Readme enhancements

[cholmes]

Fleshed out more in the readme.

Closes #10, closes #12, closes #15

[jorisvandenbossche]

Thanks! Added a bunch of questions/comments

[jorisvandenbossche]

I certainly understand the importance of those cloud data warehouses, but personally I think interoperability goes beyond that. For example, we are already using GeoParquet as a fast and interoperable format for Python and R users

[jorisvandenbossche]

Or, put differently, we are maybe missing a goal here. One of the main reasons we started using this in GeoPandas, is to have fast/efficient, columnar file format to store geospatial vector data (in addition to the traditional shapefile / geopackage / geojson). Or maybe that doesn't need to be listed explicitly? (something like "Enable the Parquet file format to store geospatial data" is already covered by the very first paragraph of this README?)

[cholmes]

I think it'd be a good goal to lay that out explicitly, like as the first goal, and make the cloud data warehouse point build on it.

Thinking something like:

• Build a fast/efficient, columnar file format to store geospatial vector data to enable XXXX

I can use help on what it enables...

[cholmes]

Ok, made an attempt at this. Ended up splitting out 'goals' from 'features', and made a stab at explaining it. I've not been deep in the new columnar workflows, so any tweaks to better explain the potential are more than welcome.

[jorisvandenbossche]

Build a fast/efficient, columnar file format to store geospatial vector data to enable XXXX

I can use help on what it enables...

I think in the first place, it enables "fast/efficient data access", I would say
(and then also other features from Parquet, like very good compression (so small file sizes), cheap reading of a subset of columns (the columnar nature), the type system (eg nested types), filtering chunks based on column statistics, ..)

But taking a look at your updates now!

[cholmes]

Ah, that's cool. Some geospatial people love really complex data structures, GML went nuts with that stuff. I think we should definitely focus on just the simple features use case and do that really well, but good to know that more complex data structures can be supported.

[cholmes]

https://databricks.com/glossary/what-is-parquet is a great overview - I think we should include a link to it somewhere in the readme. It includes a decent overview of columnar formats. I'll add it in.

[kylebarron]

I think the nested types are mostly relevant when talking about an arrow-native geometry encoding. For example a nested list of values would be able to describe a column of Points, while still having a favorable in-memory encoding (i.e. flat arrays underneath).

[cholmes]

Very cool. So we'll leave them off here for now, but maybe mention in the future.

[jorisvandenbossche]

We can indeed use those nested data types to store geometries (the arrow-native encoding proposal), but just a note that there are ecosystems where they make heavy use of the nested schemas that Parquet enables (eg I think Spark supports this quite well; I am not super familiar with it, since nested columns are not really supported in python/pandas). For example, it can map nicely to structured data you might encounter in json files, logs, etc.

(this is just a clarification, agreed it is not that important to have it included in this PR)

[jorisvandenbossche]

Thanks for all the updates, this is looking great!

[jorisvandenbossche]

I am going to merge this, so we have some more content on the repo landing page. More comments here are of course still welcome, we can always do follow-up PRs.