Newlines and lexer states
Chris Clark · ACM SIGPLAN Notices · 2000
Special tokens that must occur at the start of a line are a common problem.So common that LEX uses the caret character (^) to mark the start of the line in regular expressions.Many cases are simple enough that this is an adequate solution.That is little consolation for those users of the many lexer generators that do not have built in start-of-line support.Nor does it help when the requirements are more complex.For those cases, it is worth investigating and understanding this area more.