From looking at their source briefly, I can make two observations: Firstly, thei... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		burntsushi on June 8, 2016 \| parent \| context \| favorite \| on: ParaText: CSV parsing at 2.5 GB per second From looking at their source briefly, I can make two observations: Firstly, their core parsing routine is a state machine not unlike the one found in rust-csv: https://github.com/wiseio/paratext/blob/master/src/csv/rowba... Secondly, they do indeed appear to be achieving high parsing speed using coarsely grained parallelism. You can see the code that chunks up the CSV data here: https://github.com/wiseio/paratext/blob/master/src/generic/c... It's pretty clever!

Ygg2 on June 8, 2016 [–]

Can this be used to speed up rust-csv?

burntsushi on June 8, 2016 | [–]

It feels like something that should be built on top of rust-csv.

Ygg2 on June 9, 2016 | | [–]

I thought this was base ability of the parser to parse CSV in chunks.

Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact