syn
v0.0.36Lexer, streaming parser, Ast views, and diagnostics toolkit
riot add syn syn = "0.0.36" - Last release
- May 11, 2026
- License
- Apache-2.0
- Dependencies
- 3
- Dependents
- 5
- Installs
- 101
- Published versions
- 26
- Owner
-
@leostera
README Expand Collapse
syn
syn is Riot's lossless OCaml lexer, streaming parser, diagnostics, and typed
Ast view layer.
It exists for tooling, not compilation. The package keeps source spans, raw tokens, comments, docstrings, diagnostics, and a lossless syntax tree available from one parser path. Typed Ast modules are lightweight views over that tree.
Current shape
syn works in layers:
Lexertokenizes source code for debugging and token-oriented tools.Parserconsumes anIO.IoVec.IoSlice.tand streams parser events.SyntaxTreebuilds the lossless tree from raw tokens and parser events.Astexposes typed views over the same lossless tree.Depsextracts syntactic module dependencies from Ast views.
Two invariants matter:
- parsing always returns a syntax tree, even for malformed input
- typed Ast views do not allocate a second concrete tree
In other words:
let source =
IO.IoVec.IoSlice.from_string contents
|> Result.expect ~msg:"failed to create source slice"
in
let result = Syn.parse ~filename:(Path.v "file.ml") source in
if Vector.length result.Syn.Parser.diagnostics = 0 then
let root = Syn.Ast.SourceFile.make result.Syn.Parser.tree in
ignore root
else
ignore result.Syn.Parser.diagnostics
Public API
The high-level entrypoints are:
Syn.tokenizeSyn.parseSyn.parse_implementationSyn.parse_interface
Syn.parse* APIs accept IO.IoVec.IoSlice.t. Keep string-to-slice conversion
at the edge of callers so the parser can share source storage with downstream
tools.
Syn.parse returns a Parser.parse_result with:
source: the original source slicekind: implementation or interfacetokens: source-backed raw token streamtree: vector-backed lossless syntax treediagnostics: structured parse diagnostics
Use Syn.SyntaxTree when you need raw lossless traversal. Use Syn.Ast when
you want grammar-oriented typed views over the same tree.
CLI helpers
The package ships CLI surfaces used by the fixture suite:
riot run syn -- parse --json path/to/file.ml
riot run syn -- token-stream --json path/to/file.ml
parse --json prints the lossless tree and diagnostics. token-stream --json
prints token records, including leading trivia.
Testing
The main validation surfaces are:
timeout 120 riot build -p syn --json
timeout 180 riot test -p syn -f deps --json
timeout 180 riot test -p syn -f fixture --json
timeout 180 riot test -p syn -f diagnostic --json
timeout 180 riot test -p syn -f ast --json
When extending syn, add parser and Ast coverage before relying on new syntax
from downstream packages.
syn
syn is Riot's lossless OCaml lexer, streaming parser, diagnostics, and typed
Ast view layer.
It exists for tooling, not compilation. The package keeps source spans, raw tokens, comments, docstrings, diagnostics, and a lossless syntax tree available from one parser path. Typed Ast modules are lightweight views over that tree.
Current shape
syn works in layers:
Lexertokenizes source code for debugging and token-oriented tools.Parserconsumes anIO.IoVec.IoSlice.tand streams parser events.SyntaxTreebuilds the lossless tree from raw tokens and parser events.Astexposes typed views over the same lossless tree.Depsextracts syntactic module dependencies from Ast views.
Two invariants matter:
- parsing always returns a syntax tree, even for malformed input
- typed Ast views do not allocate a second concrete tree
In other words:
let source =
IO.IoVec.IoSlice.from_string contents
|> Result.expect ~msg:"failed to create source slice"
in
let result = Syn.parse ~filename:(Path.v "file.ml") source in
if Vector.length result.Syn.Parser.diagnostics = 0 then
let root = Syn.Ast.SourceFile.make result.Syn.Parser.tree in
ignore root
else
ignore result.Syn.Parser.diagnostics
Public API
The high-level entrypoints are:
Syn.tokenizeSyn.parseSyn.parse_implementationSyn.parse_interface
Syn.parse* APIs accept IO.IoVec.IoSlice.t. Keep string-to-slice conversion
at the edge of callers so the parser can share source storage with downstream
tools.
Syn.parse returns a Parser.parse_result with:
source: the original source slicekind: implementation or interfacetokens: source-backed raw token streamtree: vector-backed lossless syntax treediagnostics: structured parse diagnostics
Use Syn.SyntaxTree when you need raw lossless traversal. Use Syn.Ast when
you want grammar-oriented typed views over the same tree.
CLI helpers
The package ships CLI surfaces used by the fixture suite:
riot run syn -- parse --json path/to/file.ml
riot run syn -- token-stream --json path/to/file.ml
parse --json prints the lossless tree and diagnostics. token-stream --json
prints token records, including leading trivia.
Testing
The main validation surfaces are:
timeout 120 riot build -p syn --json
timeout 180 riot test -p syn -f deps --json
timeout 180 riot test -p syn -f fixture --json
timeout 180 riot test -p syn -f diagnostic --json
timeout 180 riot test -p syn -f ast --json
When extending syn, add parser and Ast coverage before relying on new syntax
from downstream packages.