Skip to content

Commit

Permalink
Fix missing whitespace typo
Browse files Browse the repository at this point in the history
  • Loading branch information
Laureano authored Jan 17, 2023
1 parent e5a37de commit 1d83f5c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Splitting an MRF JSON document into NDJSON using `jsplit` takes time. `jsplit` m
See the models in [`models/mrf.go`](pkg/mrfparse/models/mrf.go) for the parquet schema.

## How the core parser works
An MRF file is split into a set of JSON documents using a fork of [`jsplit`](https://github.com/dolthub/jsplit) that has been modified to supportreading and writing to cloud storage and use as a Go module. `jsplit` generates a root document and set of `provider-reference` and `in-network-rates` files. These files are in NDJSON format, allowing them to be consumed memory efficently. They are parsed line by line using [`simdjson-go`](https://github.com/minio/simdjson-go) and output to a parquet dataset.
An MRF file is split into a set of JSON documents using a fork of [`jsplit`](https://github.com/dolthub/jsplit) that has been modified to support reading and writing to cloud storage and use as a Go module. `jsplit` generates a root document and set of `provider-reference` and `in-network-rates` files. These files are in NDJSON format, allowing them to be consumed memory efficently. They are parsed line by line using [`simdjson-go`](https://github.com/minio/simdjson-go) and output to a parquet dataset.

`in-network-rates` files are parsed first, allowing us to filter against our `services` list and build up a list of providers for whom we have pricing data. This provider list is then used to filter the `provider-reference` files.

Expand Down

0 comments on commit 1d83f5c

Please sign in to comment.