The generated structural analyzer must read a sequence of characters from a file and build the corresponding abstract syntax tree. Eli can extract the definitions of literal basic symbols from the grammars, but neither Section 1.2 nor Figure 1 describes the sequence of characters making up identifiers and numbers. There is also no indication of what constitutes a comment.
Figure 6
contains the necessary definitions. The Eli library provides canned descriptions (such as PASCAL_IDENTIFIER) for all of these character sequences. (White space will be automatically ignored by an Eli-generated scanner unless the user explicitly defines some other behavior.)The canned descriptions of PASCAL_INTEGER and PASCAL_REAL do not guarantee that each distinct denotation has a unique internal representation no matter how many times it appears in the source text. It is convenient to have unique representations for denotations, and hence Figure 6 overrides the creation of their internal representations. The token processor mkidn enters the scanned character sequence into the string table if and only if it is not already in that table. It then uses the index of the (newly or previously) stored string as the internal representation of the basic symbol.