Parse Tokens With Spans Using chumsky In Rust
```cargo
[dependencies]
chumsky = { version = "0.9.2" }
```
use chumsky::prelude::*;
use std::ops::Range;
#[derive(Debug)]
pub enum Token {
LetterA,
LetterB,
}
fn main() {
let source = "abba";
let result = letters().parse(source);
dbg!(&result);
}
fn letters() -> impl Parser<char, Vec<(Token, Range<usize>)>, Error = Simple<char>> {
letter1().or(letter2()).repeated()
}
fn letter1() -> impl Parser<char, (Token, Range<usize>), Error = Simple<char>> {
just("a").map_with_span(|_, span| (Token::LetterA, span))
}
fn letter2() -> impl Parser<char, (Token, Range<usize>), Error = Simple<char>> {
just("b").map_with_span(|_, span| (Token::LetterB, span))
}
Output:
[/Users/alan/.cargo/target/55/19854259915251/_active_nvim_run:18:5] &result = Ok(
[
(
LetterA,
0..1,
),
(
LetterB,
1..2,
),
(
LetterB,
2..3,
),
(
LetterA,
3..4,
),
],
)
Notes
- This is a test I'm doing for the Neopolitan LSP
-
Individual Token are created with their spans in
letter1
andletter2
then assembled into a Vec inletters
-- end of line --