Latest @ApacheParquet for .NET introduces support for appending data to existing files and reading into CSV with type inferring. This is great news for big data scenarios because now you can generate massive files without spending much RAM by streaming data in chunks.
We've been working hard on Parquet.NET to give developers high level abstractions over Parquet so that there is an easy entrypoint into developing with Parquet that is not onerous for developers new to the format. As such, we've created ParquetConvert which allows the trivial creation of Parquet files from IEnumerable<T> Check out this gist I created that shows how to do this. It shows that a simple class as below can be serialized so easily that for the consuming developer there is next to no code. With a simple call to ParquetConvert.Serialize, we can save a collection of these elements in Parquet. Inspecting with Parq shows our output: We'll be continuing to add more features to help developers maximise their productivity with Parquet, as well as retaining low level features that allow complete control of the Parquet format from dotnet.