Lexers (also called tokenizers) can be used to prepare a string or stream of character data into a form suitable as an input for parsers generated by parser generators.

  • cl-lex - cl-lex is a set of Common Lisp macros for generating lexical analyzers automatically
  • DEFLEXER - The LEXER package implements a lexical-analyzer-generator called DEFLEXER, which is built on top of both REGEX and CLAWK
  • graylex - graylex offers a means to do string operations on input streams without slurping all input at once by using Common Lisp Gray Streams, fixed-sized and flexible buffers
  • token-stream - Lexer class for cl-stream
  • Zebu - A Tool for Specifying Reversible LALR(1) Parsers

The LEXER Package

The lexer package is a tokenizer for Common Lisp that makes heavy use of the re package.

Repository: https://github.com/massung/lexer

License: Apache 2