Reads the compiler's token stream as a FunctionalScript module: import statements, then ordinary and exported const statements, let statements, assignments and while and classic for loops, with an optional final export default or terminal throw. A loop ends at its closing }. Other statements end with ;, or where JavaScript inserts one; a module that does not throw needs at least one export.
It is the upper layer of a layered parser: the tokenizer turns code points into tokens, and this turns tokens into a ResolvedModule. Both layers are an LL(1) grammar read by fjs/ebnf/ll1 and a fold over what the grammar matched; only the alphabet differs — code points there, token symbols here. This layer is two modules along the line its two questions draw: ./syntax holds the rewrite set, which turns the grammar's tree into the syntax tree — a node per value, a record per statement — and parseSyntax, the reader over tokens; ./module.f.mjs is the fold, which resolves the tree's names into the AST and refuses what only a word or a line can tell, and parseFromTokens, the two in sequence. Trivia is in neither alphabet's stream for long: the tokenizer reads it and leaves it out, keeping of it the one fact a rule of the language asks, whether a newline stood before a token (fjs/compiler/tokenizer).
parseFromTokens returns one record with imports, a structured source body, and complete source binding evidence. Source values refer to original declaration identities; blocks retain initialization plans, assignments, plain loops and terminal branches for the common sugar pass. Each loop retains its complete condition, optional step and body event tree, including nested source-function reads. The parser admits zero or several outer writes without selecting loop state; sugar intersects those events with established bindings and the shared isMutable facts. Loop-local names remain separate resolved identities. Classic for header declarations are resolved in a scope surrounding the condition, step and body; the step is outside the body's nested lexical scope. scope records initialization, reads, writes, execution regions and ordinary source function boundaries. bindings is the result of one shared analyzeBindings pass after that complete scope exists; origins retains each binding's name, declaration kind and source position. Open scopes append their trace events with immutable lists, so another read does not copy all preceding reads. Closing a function or region materializes its public event array once; a separate count preserves source target indices.
Each declaration gets its own identity even when its initializer aliases an existing value. A snapshot of a loop variable is therefore a different source binding, and later writes to the variable do not make the snapshot mutable. Pattern expansion gives destructured locals their own identities; parameter and destructuring initialization is not reassignment. Loop assignments retain the original source identity across the value versions used by graph lowering.
Guard and loop lexical regions retain their owning source function target. They create no source capture boundary. An ordinary nested function does, even when it is called immediately. Whole-helper recognition retains its parameter and local facts before replacing the helper with an intrinsic operation.
Ordinary conditionals may finish normally or return from their owning function, with an optional else. Their lexical arms retain declarations, assignments and complete source evidence; the first admitted profile joins at most one outer binding. Branch-local writes do not count toward that limit. The condition establishes its versions before either arm; sugar supplies unchanged versions for an absent else and resumes following work once on normal paths. An early return in a lexical arm retains its function target as a source statement, including through nested guards and switches. It supplies no normal join result and skips the following computation. An ordinary nested function keeps its own return owner. This statement-return context is separate from permission for an arm's final loop to complete normally; switch cases still require an exit. Explicit throws in this branch profile remain refused, while existing terminating guards keep their behavior. Generic branch syntax forms an AST before admission checks; ordinary function bodies still require an explicit terminator.
Loop bodies and their conditional arms retain valued return and throw as source statements, including all syntax after them. A return names its owning ordinary function; lexical regions preserve that identity, while a nested function establishes another one. Module loops have no return owner and refuse return, even in an unreached arm. A module loop may throw. Name, declaration, write and capture checks still inspect statements after either exit; sugar skips their computations when the exit is selected. These exits use the same valued spelling and line-terminator checks as a function body's terminal exit.
Loop-body switches retain one discriminant, ordered stacked case labels, fresh case scopes and an optional last default. Every braced arm must leave the arm: break targets that switch, continue its enclosing loop, and return its ordinary function. Conditional arms and nested switches keep those owners. Normal case fallthrough is refused after complete name and declaration checks; the completion walk uses an explicit stack and shared continuations. Sugar retains selected state results through the existing one-pass loop representation. Existing terminal switches outside loop bodies retain their syntax and rules.
Module loops enter the same resolution worklist and lexical loop regions as body loops. Their completed source packet resumes the module declaration/export fold without introducing a function or a value-region write boundary. Statements and named exports retain their written initialization order; sugar uses the same state/result path and keeps earlier exported constants as snapshots.
These facts decide the capture policy: a variable assigned anywhere in its scope, a let's or a parameter's, is refused to a function held as a value, while a function called where it is written, by a plain or an optional call, keeps reading it. Direct immediate calls that normalize to expression blocks may also assign enclosing variables; normalization exports their updated immutable binding versions to the continuation. A call that retains an ordinary invocation boundary cannot write an enclosing variable. A variable assigned nowhere is captured as a const is (spec: variables). The fold records each reference across a function boundary and decides it when the function closes; the shared analysis says, at the end, which bindings were reassigned. fjs/js/ast/sugar owns pattern planning, invocation layout, defaults, branch normalization and immutable assignment versions. Its pattern planner runs source admission callbacks during one ordered walk before shared binding analysis.
capturedBy names source functions containing a direct resolved read or write. Intermediate frame forwarding for a deeper closure is a graph-construction concern and adds no synthetic source reads.
./grammar holds it:
module ::= import* moduleStatement* last eof
moduleStatement ::= const | let | while | forStatement | bodyNameStatement
import ::= 'import' clause 'from' string [ 'with' '{' id ':' string '}' ] end
clause ::= named | id [ ',' named ]
named ::= '{' [ items(binding) ] '}'
binding ::= id [ 'as' id ]
const ::= 'const' target '=' value end
let ::= 'let' target '=' value end
target ::= id | array | object
last ::= 'export' ( 'default' value end | const moduleStatement* [ last ] )
| 'throw' value end
end ::= [ ';' ]
while ::= 'while' '(' value ')' loopBlock
value ::= '-' unaryOperand tail | '~' unaryOperand tail
| '!' unaryOperand tail | 'typeof' unaryOperand tail
| (primitive | template | array | object) access* powTail tail
| id arrowOrRest
| '(' (func | value afterValue)
body ::= '-' unaryOperand tail | '~' unaryOperand tail
| '!' unaryOperand tail | 'typeof' unaryOperand tail
| (primitive | template | array) access* powTail tail
| id arrowOrRest
| '(' (func | value afterValue) | block
unary ::= '-' unaryOperand | '~' unaryOperand | '!' unaryOperand
| 'typeof' unaryOperand
| (primitive | template | id | array | object) access* powTail
| '(' group
unaryOperand ::= '-' unaryOperand | '~' unaryOperand | '!' unaryOperand
| 'typeof' unaryOperand
| (primitive | template | id | array | object) access*
| '(' groupOperand
template ::= noSubstitutionTemplate
| templateHead value { templateMiddle value } templateTail
block ::= '{' statement* terminator '}'
statement ::= const | ifStatement | let
| whileStatement | forStatement | bodyNameStatement
ifStatement ::= 'if' '(' value ')' branchBlock [ 'else' branchElse ]
branchBlock ::= '{' statement* [ terminator ] '}'
branchElse ::= branchBlock | ifStatement
terminator ::= 'return' value end | 'throw' value end
| 'switch' '(' value ')' '{' 'case' value ':' caseTail
caseTail ::= 'case' value ':' caseTail | block afterCase
afterCase ::= 'case' value ':' caseTail | 'default' ':' block '}'
| '}' statement* terminator
loopBlock ::= '{' loopStatement* '}'
loopStatement ::= const | let | bodyNameStatement | whileStatement | forStatement
| loopIfStatement | 'break' end | 'continue' end
| 'return' value end | 'throw' value end | loopSwitch
loopSwitch ::= 'switch' '(' value ')' '{' loopCase+ [ 'default' ':' loopBlock ] '}'
loopCase ::= ( 'case' value ':' )+ loopBlock
whileStatement ::= 'while' '(' value ')' loopBlock
forStatement ::= 'for' '(' [ forInitializer ] ';' [ value ] ';' [ nameExpression ] ')' loopBlock
forInitializer ::= ( 'const' | 'let' ) target '=' value | nameExpression
loopIfStatement ::= 'if' '(' value ')' loopBlock [ 'else' loopElse ]
loopElse ::= loopBlock | loopIfStatement
nameExpression ::= id ( assignmentOperator value | <the rest of the value the name begins> )
bodyNameStatement ::= nameExpression end
func ::= [ '...' id ] ')' '=>' body
afterValue ::= ',' [ parenItems ] ')' arrowOrRest | ')' arrowOrRest
| '=' value [ ',' [ names ] ] ')' '=>' body
arrowOrRest ::= '=>' body | access* powTail tail
parenItems ::= '...' id | value [ '=' value ] [ ',' [ parenItems ] ]
names ::= '...' id | target [ '=' value ] [ ',' [ names ] ]
group ::= value { ',' value } ')' access* powTail
groupOperand ::= value { ',' value } ')' access*
powTail ::= [ '**' unary ]
eagerTail ::= { mulOp unary }
{ addOp unary <the multiplicative repeat above> }
…six more layers, each repeating over every layer below it the
same way — shift, relational, equality, bitwiseAnd, bitwiseXor,
bitwiseOr, JavaScript's own order
logicalAndRound ::= '&&' unary eagerTail
logicalOrRound ::= '||' unary eagerTail { logicalAndRound }
nullishRound ::= '??' unary eagerTail
circuitTail ::= [ logicalAndRound { logicalAndRound } { logicalOrRound }
| logicalOrRound { logicalOrRound }
| nullishRound { nullishRound } ]
conditionalTail ::= [ '?' value ':' value ]
tail ::= eagerTail circuitTail conditionalTail
access ::= '.' id | '[' (string | number) ']' | '(' [ items(value) ] ')'
| '?.' optionalStep
optionalStep ::= id | '[' (string | number) ']' | '(' [ items(value) ] ')'
array ::= '[' [ items(value) ] ']'
object ::= '{' [ items(entry) ] '}'
entry ::= '...' value | member
member ::= id [ ':' value ] | string ':' value | '[' string ']' ':' value
items ::= item [ ',' [ items ] ]
It is LL(1): one symbol of lookahead decides every choice, and the backend refuses a grammar where it would not, before any input.
A ( opens an empty/rest-only function or a shared expression/parameter prefix. afterValue factors the comma, the closing parenthesis and the = of the first parameter's default; arrowOrRest then selects the arrow or the ordinary expression continuation after the parenthesis. The binding pass requires a binding target — a name, or a pattern — where that prefix becomes a parameter, rejecting (a + b) => 1 while preserving (a + b). After the comma the items are read the same way, parenItems, a cover of the parameters and of the comma operator's operands, which agree up to the ): (a, b) => 1 is a function and (a, b) a comma (spec: comma). The fold refuses an item that is no binding target where => follows, and a default, a rest or a trailing comma where it does not, at the =, the ... or the ). A group no function may open, under a prefix or as an operator's operand, reads a comma's operands as values outright, group's { ',' value }. Bare a => a uses the same arrow continuation. The fixed parameters after the first are values the fold reads as targets, each with an optional default, = value — targets outright after a default, names — and a final rest name is allowed after them; it takes no default and cannot be followed by another parameter or a comma. The parameters from the first default on are the fold's to read from the rest array, landing in the function's own scope before its body (spec: functions). The no-newline rule before => is the fold's, not the grammar's: trivia is no symbol of the grammar, and the => token says whether a line break stood before it, a line comment's end or a block comment holding one included.
A prefix takes the group under its ( and not paren, the two differing by the function: -(...a) => 1 is a syntax error in JavaScript and -((...a) => 1) is not, so the operand is the group alone and the ... is refused where JavaScript refuses it rather than at the (. Every binary operator's operand is unary — see the next section for why it can be no wider a rule — but a prefix's own operand is unaryOperand, a narrower rule still: JavaScript refuses ** immediately after a unary-prefixed operand, full stop, at any depth (- -2 ** 2 exactly as - 2 ** 2), so unaryOperand is every alternative unary has minus powTail, recursing through itself rather than unary for a nested prefix. Only (-2) ** 2 and -(2 ** 2) write either reading: parentheses that move the ** to where it no longer immediately follows the prefix.
tail, the operator suffix — Stage A's eager ladder and Stage B's lazy operators and conditional above it (spec/operators/README.md) — is threaded onto every branch of value/body that may carry one, inline, rather than wrapping a shared primary the way a textbook precedence ladder would. That wrapping was tried first and rejected: func's body is unbounded, reading everything to its right as its own, so a primary the ladder also wrapped would leak the ladder's own follow set down into the body and manufacture an LL(1) conflict with no real ambiguity behind it — a greedy reader never needs the choice the checker flags, but the checker cannot see that. Spelling tail inline, with unary — narrow, func excluded — as every operand throughout, avoids the leak entirely: func is reachable only where value/body put it directly, never as a repeated operand any layer wraps.
The short-circuit level, circuitTail, is a choice its first operator makes rather than one more repeat: JavaScript keeps ?? apart from &&/|| at one nesting by giving the two their own productions, LogicalORExpression beside CoalesceExpression, and spelled as that choice the two alternatives open with one operand, a first/first conflict the checker refuses before any input. So the operand belongs to the branch its operator opens, the choice is made at that operator — one symbol — and a chain committed to &&/|| has no round for ??, nor a ?? chain for either: a ?? b || c fails at the ||, refused by the grammar's shape and by nothing after it. The conditional is the top, ? value : value, each arm the whole value rule — JavaScript's arms are AssignmentExpressions, and with no assignment the ladder's own top is the nearest — so a nested conditional associates to the right through the arms' recursion, and : follows a function's body there without a conflict, nothing a body may continue with beginning with it: a ? () => 1 : 2 is the function and the else arm, as JavaScript reads it. Both are read by the same fold as the eager layers, a branch's round being a layer's round and its continuation the repeat lists a value's own tail is, plus one reader for the two arms.
Three more things are spelled for one symbol of lookahead, each a conflict the backtracking grammar this replaced had (the record of all eight):
- Trivia is no symbol. Whitespace, newlines and comments are not in the stream the grammar reads:
fjs/compiler/tokenizerleaves them out and marks each token with whether a newline stood before it,newlineinModuleTokenWithMetadata. So no rule mentions trivia and none begins with it — the backtracking grammar's statement terminator and its final optional;both did, a first/first conflict on the trivia symbols — and JavaScript's three line-break rules are facts of a token the fold checks rather than shapes the grammar spells:[no LineTerminator here]before=>and afterreturnorthrow, refused at the=>and at the value's first token, and automatic semicolon insertion, below. - A statement ends at
;, or at nothing. A newline is never the symbol it ends at, and telling a newline from a;reached through newlines took unbounded lookahead while it was one. So the;is optional and the grammar looks no further — one symbol still decides, since;begins no statement and no statement's continuation. JavaScript's rule, that a;is inserted before a token on a new line and not before one on the same line, is then the same fact of the token: the fold refuses a statement whose predecessor omitted its;at its first token unless that token began a line. Which is why the refusal is the fold's and not the grammar's: the grammar readsconst a = 1 export default a;, andfoldModuleanswersunexpected tokenat theexport, as the grammar did when the;was required (spec/README.md). DataJS still requires the;, in its own reader. - A list is right-recursive. After an item and its comma, the lookahead says whether an item or the closing bracket follows, so a trailing comma is a comma nothing follows.
Two rules that were once code are shape. Statement ordering — every import before every ordinary statement or exported const — is in the grammar, and a late import is a token it cannot use. The export rule factors the common keyword: default ends the module, while const can be followed by ordinary statements and another export. Its optional tail permits a named-only module. A reserved literal — true, false, null, undefined, NaN, Infinity — has its own symbol, never id's, and the rules take it wherever a name may stand: { NaN: 1 } and a.NaN are a key and an access, and const NaN = 1; reaches the fold and is refused there, as const if = 1; is. Where a value may stand it is the value it names, which is the one position the two rules keep apart. -Infinity is two tokens, the - being the unary minus the grammar reads rather than a sign folded into the word, so -Infinity is a negation where a value may stand and a - the grammar answers at where a name may. import, const and export in the wrong order report unexpected token at the offending keyword.
This is the line that decides where a check belongs, and it is sharper than "syntax versus semantics". A token's text rides along as metadata, invisible to a grammar whose terminals are symbols. So every check that has to read a word is the fold's:
- an identifier naming no
constorimport; - a
constorimportname already bound — they share one map, so a name taken by either is taken for both; - a JavaScript keyword where JavaScript wants an identifier — a name bound or referenced. The tokenizer hands every keyword over as an
idtoken, since a key or the name after.may be one, soconst if = 1;andexport default class;are the fold's to refuse and{ if: 1 }is a member: a broken JavaScript program is a broken FunctionalScript program.typeofopens a value, so the grammar keeps it out ofidentifier, the reference rule, where it would conflict with its own prefix branch; a binding takes the wideridentifierName, soconst typeof = 1;still reaches the fold, and{ typeof: 1 }anda.typeofstay members; - a reserved conversion global,
NumberorString, bound or referenced anywhere but as the callee ofNumber(x)orString(x)(spec: number conversion, string conversion) —fjs/js/keywords'reservedGlobals, a list beside the keywords rather than among them, since JavaScript has no such keyword and its tokenizer keeps the word anid. The fold reads the callee's word before resolving it, so the word alone decides, no scope ever binding it.Number()is the literal0, andString()is the empty-string literal. Multiple arguments and spreads are refused at the callee withNumber takes one argumentorString takes one argument, by design: a conversion is an operator, with a fixed set of operands (spec: operators and functions). Guarded calls, by design too, standalone references, and namespace members other than the static functions below are refused as reserved-word uses. A key or property name remains ordinary,{ String: 1 }.Stringbeing valid; - the reserved globals
ObjectandNumber, bound or referenced anywhere but as the namespace of a static function —Object.isand the fourNumberpredicates,Number.isInteger— besideNumber's conversion andObjectin theentryhelper below (spec: built-in static functions). The read of a static function, in either key spelling, is the AST's value of its name,['Object.is'], a chain's steps after it going on over that value; a guard on the namespace,Object?.is, is refused at the word, a namespace being no value to guard; an object pattern over the bare word reads each key as that access does, soconst { is } = Object;isconst is = Object.is;and the namespace, no value, is never resolved. Which members a namespace has is the EDAG'sstaticIds. Any other member, a computed key, the word alone and any other pattern over it are refused as reserved-word uses; - an import attribute other than
type: "json", the one JavaScript defines, read from the key's and the value's words; - a body
constthat takes a name the body already binds, its parameter included, which is a duplicate as a module's is. A name a scope around the body binds is not: the body'sconstshadows it, as in JavaScript — unless the body has already read that name from outside, before theconstor in its own initializer, which iscapture shadowed: not a rule of the language but a forward reference inside a body, not yet supported (todo/body-const-forward-reference.md), refused because JavaScript would read the body'sconstthere; - a bare or string
__proto__key, which JavaScript reads as an instruction to replace the prototype. The computed spelling{ ["__proto__"]: v }denotes an ordinary property and is accepted, so this is not a lexical rule either; - a binding pattern,
const { a, b: [c] } = v;or({ a, b: [c] }) => a(spec: destructuring): the grammar reads the array or object literal the same text spells, and the reader keeps the pattern reading of it beside the node, so the fold is what refuses a value in a binding position (invalid destructuring target) — a literal, an access, a call, or a group,[(a)], JavaScript's early error — a name bound twice in one pattern or across a parameter list, a key a prototype gives a value, and what the language does not read yet, a rest element and a computed key beyond a literal alone in its brackets, each by name — and a module whose exports are all patterns binding no name, which exports nothing where a name was owed (module exports nothing). The common sugar planner records each expanded read and its original identity, with the fold supplying name and key policy callbacks. Executable definitions and parameter layout are chosen by sugar after source binding analysis; - an access naming a property of a built-in prototype,
a.push,a["toString"]ora.__proto__in either spelling — every namefjs/js/prototypelists butlength, which a value owns — since an access reads an own property and JavaScript would read the prototype's, as spec: property accessor prohibits. The key of an access is a constant — an identifier after., a string or a number in[ ]— or the conversiona[Number(i)], whose number names no prototype property, so what remains is the EDAG's own form,['.', base, key]. The grammar reads any value in brackets, and the fold refuses every other at the token it begins with,a[i]included (computed key is not Number(...)), after the base, as JavaScript evaluates the two; - a method call naming a member function a module may not call,
a.push(1)ora.valueOf()— the namesprohibitedCallsin the same module lists, its README saying why for each. An access that is a call's callee, through a group as well, is checked against that list instead of the read rule, soa.at(0)anda.toString()are calls like any other whilea.atstays a refused read: a detached built-in is a function that only fails; - the
entryandhasEntryhelpers, which are matched whole rather than resolved (spec: entry): a function of two parameters whose block body binds the descriptorObject.getOwnPropertyDescriptoranswers for them and returns the value an enumerable one holds,x?.enumerable ? x.value : undefined, or its presence,x?.enumerable === true— under any three distinct names, the keys in either spelling, with or without its semicolons where JavaScript inserts them — is the AST's['entry']or['hasEntry'];Objectis reserved, so no scope binds the word and the shape alone decides. A function that is neither helper, by shape or by its names, is resolved as every function is, whereObjectis refused as the reserved word it is; - an intrinsic's helper, matched whole the same way: a function of as many bare, distinct names as the intrinsic's
lengthwhose expression body is the one member call of the intrinsic's name on the first with the rest,(s, a, b) => s.slice(a, b)— the AST's['slice'], by the member's name, the listintrinsicsinfjs/edagand the helpersfjs/js/intrinsic. A step from the shape is an ordinary function with an ordinary member call.
The fold is where a symbol table already exists: ['ref', bindingId] names one resolved source declaration. The grammar does not approximate this lookup.
A reference in a function body resolves its own parameters and prior locals before walking outward. The resolver remembers names read through intermediate scopes for the existing capture-shadow admission rule. Shared capturedBy facts record direct references from source functions; sugar constructs capture forwarding and snapshots the current immutable value of each source identity. Graph construction then compacts equal capture values into frame slots.
A const value is resolved before its name is initialized, so const a = a; is const not found, as required by JavaScript's declaration ordering. A function that is the whole initializer of one name retains that declaration as its self identity, unless a parameter or local shadows it. A function nested inside it resolves the same declaration identity. The name is the function's alone: const f = [() => f]; reads no function's own name. A later local of a name already read from outside remains capture shadowed.
The backend folds a rewrite set into the parse: a mapping per rule, applied as the rule's node comes into existence, with the rules under it already mapped (fjs/ebnf/ll1). A mapping sees one node and no environment, so what the mappings build is a module of nodes — a primitive converted from its token, a reference by the identifier token that spells it, a container of nodes, a record per statement — and the names are resolved after the grammar has matched the whole module, statement by statement, on the pattern fjs/media/datajs set.
The syntax reader retains function declarations, plain loop statements, terminal guards and switches as source constructs. Resolution preserves them as checked source blocks and original binding references. () => 7 and () => { return 7; } resolve to the same targeted return. A guard retains its condition and two lexical blocks: the written block and the remaining statements. A switch retains the discriminant, ordered labels, cases and final fallback. A loop retains its condition, lexical body and ordered source binding events. Unlabeled continue resolves to that loop's target. A break resolves to the nearest loop or loop-body switch; nested regions preserve their distinct owners, including in either conditional arm. All later source statements still resolve, preserving name, write and capture checks even after an unconditional jump. The common sugar pass selects state from those events and the shared binding facts; the parser creates no helper function or recursive call.
The common sugar pass expands these records into normalized FAST regions, immutable definitions and explicit loop transfers. The grammar still requires function-body statements followed by a terminating return, throw or switch, and a module ends in exports or a throw. Source admission and ASI rules remain the same.
parseSyntax, in ./syntax, exposes that tree for proofs before the fold. It does not establish binding validity, JavaScript early errors or FunctionalScript admission; parseFromTokens remains the checked compilation entry point.
An untagged template is a primary expression. The shared tokenizer supplies cooked head, middle and tail tokens, preserving object and function braces inside full-expression substitutions. The compiler tokenizer checks the source escape policy before these chunks reach the syntax reader. The reader lowers each substitution to an internal unary String node and joins it and every chunk with left-associated binary +, including empty chunks. The fold maps that conversion to the existing AST operation, independently of source names. A template without substitutions is its cooked string. Tagged templates have no suffix production, so the grammar refuses them where their template starts.
Nothing walks the tree. The machine's own stack is on the heap, so nesting depth is the input's; a list's mapping puts one item before the list its tail's mapping already returned, so a list of any length costs one step at each of its nodes, though the tree is as deep as the list is long; and the resolution walks a value over an explicit stack of frames. The wide and deep inputs in the proof — twenty thousand siblings, twenty thousand levels — are the bar.
A tree this recursive is more than tsc unrolls in places. The list rule's type leaves the rest of the list as Rule, since a reader takes the rest from the list's own mapping, one symbol by then, rather than from the type. Every mapping is typed from its rule; the one reader the two list mappings share is typed by the shape of a list node, as a reader of a combinator's scaffolding is. The nodes the mappings build, and the alphabet they return them in, are public types in ./types.ts, as the rewrite set is.
The tokenizer emits import, const, export, default, from, with, return, throw, if and as as id tokens carrying the word in value. An alphabet keyed on a token's kind would give them all the symbol of any other identifier, and the grammar could not tell export default from two arbitrary names — module framing would be inexpressible, and so would a block body's return.
They therefore get their own names in the alphabet, which a registered mapping allows because a symbol comes from a name's position in a list and a name has no length limit.
Splitting them off obliges the grammar to provide identifier rules. Wherever an identifier is accepted the terminal is a union of id and the keyword symbols, and only the positions that frame something demand a specific keyword. There are two such unions, drawn where JavaScript draws them:
identifier,idand the seven framing keywords, for a reference — where a value may stand, and where a word that denotes a value is that value rather than a name;identifierName, those and the six words that denote a value, for a name — an object key, the name after., and a binding name, which is JavaScript'sIdentifierNameand admits every reserved word.
Which of them a position may hold is the fold's question, as it is for every other keyword, since it is a property of the word: { from: 2, return: 3 }, a.with, { NaN: 1 } and a.NaN parse, while const export = 1; and const NaN = 1; are refused where const if = 1; is. Giving a word its own symbol narrows where it is required, never where it is allowed.
Symbols come from fjs/ebnf/token_symbol, which assigns each name the symbol at its index in one list, above the Unicode range.
The alternative — deriving a symbol from the name's own bytes, so no ordered list exists — needs a much wider symbol domain and the migrations behind it. It buys a property this parser cannot observe: the encoding is built at construction, used for one parse, and no symbol is ever serialized, so order-dependence costs nothing.
Revisit if a token symbol ever has to be written to a file, which is the trigger token_symbol's own README names.
ParseError carries metadata — the position a reader is pointed at — and an optional end that extends it into a span. Only a lexical error has one: the tokenizer knows how far an unterminated string or comment runs, and splitEof passes that span through. A grammar failure points at a single token and stays a point, because a token's extent is not recorded — every token's metadata is its start alone.
Recording it for every token is the widening this design leaves undone, and the argument for it is the parser's own: the grammar matches rules over whole tokens, so every rule it reduces has a first and a last token and therefore a natural span. export default <value> is a span, not a point. The moment a formatter wants to underline a rule rather than a character, every token needs an end, and the change belongs in JsTokenWithMetadata.metadata rather than in more special cases beside ErrorToken.end. Nothing wants that yet: errorLocation in ./module.f.mjs renders the span an error already carries and the point when there is none.
One difference from the backtracking parser this replaced, deliberate and pinned by proof at the time: a newline no longer ended a statement, so export default 1 alone was unexpected end at the end of input, and const a = 1 followed by export default a; on the next line was unexpected token at export. Both are modules again since the ; became optional — but by JavaScript's rule, read by the fold from the token after the omitted ;, never by the grammar reading a newline as a terminator, which is what the port removed. Every other expectation of the parser's proof — values, positions, the order errors are reported in, the syntax failure found before the unresolved name — was met unchanged, with the ; added to its inputs.