Parsing JSON in 500 lines of Rust

    match object(src) {
        Ok(res) => return Ok(res),
        Err(JSONParseError::NotFound) => {} // if not found, that ok
        Err(e) => return Err(e),
    }

    macro_rules! try_parse_as {
        ($f:expr) => (
            match $f(src) {
                Ok(res) => return Ok(res),
                Err(JSONParseError::NotFound) => {} // if not found, that ok
                Err(e) => return Err(e),
            }
        );
    }

    try_parse_as!(object);
    try_parse_as!(array);
    // ...

Also, even though you chose to represent numbers as f64, a correct parsing algorithm is surprisingly tricky. Fortunately `f64::parse` accepts a strict superset of JSON number grammar, so you can instead count the number of characters making the number up and feed it into the Rust standard library.

Here is one in 73 lines of F# by Jon Harrop:

https://gist.github.com/isaksky/6681cfad8ced1708a04b2eca92fc...

I guess it's faster with sudo because the regular account has some resource limits, or something? Didn't mention which system is being used (Linux? macOS? Something else?) but I can't repro on my Linux system, where performance is identical:

  % cargo run --release
  [..]
  Parsing speed: 103761177.00 Bytes/s
  Parsing speed: 103.76 MB/s
  Parsing speed: 0.10 GB/s

  % doas cargo run --release
  [..]
  Parsing speed: 105401032.21 Bytes/s
  Parsing speed: 105.40 MB/s
  Parsing speed: 0.11 GB/

If OP is running this on MacOS it could be due to macOS' code signing that runs the first time you run an executable.

I have to commend on the simplicity and clarity of thought in the write up. I could see what you were up to just by a skim through it (also thanks for the ready reference)!

I'm pretty sure you'd already know of this, but once you've written your own version, it might help to compare and take notes from a popular, well-established, benchmarked library: https://github.com/serde-rs/json.

I have no rust knowledge let me ask

- is this hard to do in 500 lines of rust? - is there a catch why a implementation in rust worth mentioning on HN?

don't get me wrong im realy intested in rust.

Parsing JSON is a Minefield: https://seriot.ch/projects/parsing_json.html

I need to work on my rust JSON parser some more[0]. I intended it handle deeply nested (malicious) objects gracefully and with little memory allocations. It's also pretty fast.

I had a goal of making it "async", so it could periodically yield for large input strings, but I'm not so sure it matters much.

Currently the API is pretty impractical to use, because of the arena structure that is set up. It could be improved

[0] https://github.com/conradludgate/sonny-jim

https://www.perlmonks.org/?node_id=995856

I have implemented 2 packages in Go for parsing json:

1. https://github.com/ezpkg/iter.json: last year, using go iterator, the core parsing code [1a] is around 200 lines

1a. https://github.com/ezpkg/ezpkg/blob/main/iter.json/parser.go

2. https://github.com/iOliverNguyen/ujson: 4 years ago, using callback, around 400 lines

Regarding the 'sudo' issue: Doing a benchmark by just running an example executable is not really recommended because there's a ton of reasons why you might get differing performance.

It's probably better to set up an actual benchmark using a crate like Criterion instead [0].

[0] https://github.com/bheisler/criterion.rs

I once (maybe a long time ago?) made a parser for JSON by:

1. Reading the entire file into RAM.

2. Providing a `const char *get_value(const char *jstring, const char *path, ...)` function with a NULL-terminated parameter list that would return the position of the value of the key at the specified path.

3. Providing a `copy_value(const char *position)` function to copy the value at the specified position.

Slow? Yup!

But, it was easy and safe[1] and used absolutely minimal RAM![2]. The recursive nature of the JSON tree also allowed the caller to use a returned value from `get_value` as the `jstring` argument in further calls to `get_value`.

I might still have a fork of it lying around somewhere.

[1] "Safe" meaning "Caller had to check for NULL return values, and ensure that NULL terminated the parameter list".

[2] GCC with `-O2` and above does proper TCO, eliminating unbounded stack growth.

What about empty string keys, duplicate keys and byte order marks?

\uFEFF{"": "value", "": null}

(Yup, there's a reason those are not recommended for use)

It'd be easy and cleaner to use miette for pretty error printing.

I bet I could rewrite that in 5000 lines.

I wonder how well it works with heavily nested files. AFAIU, this is a recursive parser.

Do not parse using strip_prefix and then to_string, you're allocating a new buffer after each token! You can use Cow<str> for the result of the "parse string value" function, as most JSON strings don't have escapes you can just return a reference to that slice of the buffer, only allocating when there's actually an escape.

In general when writing a parser you should strive to minimize allocations and backtracking.

Now try parsing Rust in 500 lines of JSON

One line of Javascript and Python.

True but if can write an LLM in 100 lines of JAX, perhaps we should just train that LLM to parse JSON?