Class Tokenizer

Yep\Tokenizer\Tokenizer implements Yep\Tokenizer\ITokenizer

Namespace: Yep\Tokenizer
Located at Tokenizer.php

Methods summary
`public`	# `__construct( array $patterns, string $flags = '' )` Tokenizer constructor Tokenizer constructor Parameters `$patterns` `$flags`
`public array`	# `tokenize( string $input )` Tokenize input Tokenize input Parameters `$input` Returns array Throws `Yep\Tokenizer\UnexpectedTokenException` Implementation of `Yep\Tokenizer\ITokenizer::tokenize()`

Constants inherited from Yep\Tokenizer\ITokenizer
`LENGTH, OFFSET, POSITION, TYPE, VALUE`