scan

using the terminals (literals and patterns) from the grammar, scan the input text and return a list of "tokens" leaves. Useful for syntax highlighting. Where more than one terminal matches, the longest takes priority, and literals take priority over patterns