Note: This site is currently "Under construction". I'm migrating to a new version of my site building software. Lots of things are in a state of disrepair as a result (for example, footnote links aren't working). It's all part of the process of building in public. Most things should still be readable though.

Don't Use "rest" With "many1" In Rust's nom Parsing Crate

Note

I'm still working to figure this out...

This one works.

Code

#![allow(unused_imports)]
use nom::branch::alt;
use nom::bytes::complete::tag;
use nom::bytes::complete::take_until;
use nom::character::complete::multispace0;
use nom::combinator::rest;
use nom::multi::many1;
use nom::sequence::terminated;
use nom::IResult;

#[derive(Debug)]
enum Section {
    Alfa,
    Bravo,
}

fn main() {
    let source = "-- alfa\n\n-- bravo";
    dbg!(sections(source).unwrap());
}

fn sections(source: &str) -> IResult<&str, Vec<Section>> {
    let (source, sections) = many1(alt((alfa, bravo)))(source)?;
    Ok((source, sections))
}

fn alfa(source: &str) -> IResult<&str, Section> {
    let (source, _section) = tag("-- alfa")(source)?;
    let (source, data) = terminated(take_until("\n\n"), tag("\n\n"))(source)?;
    Ok((source, Section::Alfa))
}

fn bravo(source: &str) -> IResult<&str, Section> {
    let (source, _section) = tag("-- bravo")(source)?;
    let (source, data) = alt((terminated(take_until("\n\n"), tag("\n\n")), rest))(source)?;
    Ok((source, Section::Bravo))
}

Earlier notes about what wasn't working

Need to do a little more verification, but I think part of the problems I've been running into with nom and the `many`` functions is because I was pulling in content below them with `rest``. The way `many1`` and `many0`` work is that the stop processing when they hit an error. By using `rest`` to slurp the rest of the content that wasn't happening.

This is some code where I figured that out

Code

use nom::branch::alt;
use nom::bytes::complete::tag;
use nom::bytes::complete::take_until;
use nom::character::complete::multispace0;
use nom::combinator::rest;
use nom::multi::many1;
use nom::sequence::terminated;
use nom::IResult;

fn main() {
    let source = "alfa bravo\n\ncharlie delta\n\necho foxtrot";
    dbg!(this_works(&source).unwrap());
    dbg!(this_fails(&source).unwrap());
}

fn this_works(source: &str) -> IResult<&str, Vec<&str>> {
    let (source, x) = many1(this_works_part_2)(source)?;
    Ok((source, x))
}

fn this_works_part_2(source: &str) -> IResult<&str, &str> {
    let (source, _) = multispace0(source)?;
    let (source, captured) = terminated(take_until("\n\n"), tag("\n\n"))(source)?;
    Ok((source, captured))
}

fn this_fails(source: &str) -> IResult<&str, Vec<&str>> {
    let (source, x) = many1(this_fails_part_2)(source)?;
    Ok((source, x))
}

fn this_fails_part_2(source: &str) -> IResult<&str, &str> {
    let (source, _) = multispace0(source)?;
    let (source, captured) = alt((terminated(take_until("\n\n"), tag("\n\n")), rest))(source)?;
    Ok((source, captured))
}

Notes

  • I don't know if `rest`` always returns true or not, but it certinaly seems like it might.

References