Survey of Parquet Reader Support for Predicate Pushdown of Nested Bbox Definition #192

jwass · 2023-12-05T14:32:52Z

jwass
Dec 5, 2023
Collaborator

To help drive the decision of the bounding box specification pull request and discussion, we want to survey existing parquet readers and query engines to understand whether they support predicate pushdown of the nested bbox definition fields (struct with member minx/miny fields).

"Predicate pushdown" is a loaded phrase that could mean different things in different contexts. For the purposes of this discussion we'll define it as the ability to use the row group summary statistics to filter out whole row groups when the proposed bbox group/struct definition is used in the filter predicate. e.g. SELECT * FROM parquet_backed_table WHERE bbox.xmin > -71.

Please add more information or other missing systems in the discussion comments. I'll keep the list here updated as we get more information. For the ones that are "no", we might want to understand the level of effort to gain support.

duckdb - Yes (as of 0.10.0) (@Maxxen)

#191 (comment)
#192 (comment)

When the PR referenced in #191 (comment) lands, this should be supported in the next duckdb release.

gdal - Yes as of 3.8.0 (@rouault)

~~Per comment at GeoParquet meetup yesterday, gdal properly filters on nested fields, but does not skip whole row groups~~

Support for Overture bbox definition included. #192 (comment)

arrow-cpp - Yes as of 15.0 (@jorisvandenbossche)

Gains support when apache/arrow#39065 is merged. (Update: Merged)
As of Arrow 15.0 per discussion at GeoParquet meetup

Presto / Trino / Athena - Yes (@jwasserman)

#192 (comment)

Spark / Databricks - Yes (@jiayuasu)

Per comment #192 (comment)

Sedona - Yes as of 1.5.1 (@jiayuasu)

Per comment #192 (comment)
Should be supported in next release.
Supported as of Sedona 1.5.1

Snowflake Yes (@jiayuasu)

#192 (comment)

The column extract needs to be cast as e.g. t.value:bbox:minX::double > 41.123

BigQuery - Yes (@jatorre)

Comment: #192 (comment)

Impala - Yes (@csringhofer)

Comment: #192 (comment)
Supported on read. Impala cannot yet write complex/nested types.

Synapse ??

DataFusion - No (@kylebarron)

#191 (comment)

cudf - No (@paleolimbot)

#191 (comment)

rouault · 2023-12-05T14:42:48Z

rouault
Dec 5, 2023

gdal - No (@jorisvandenbossche)

Per comment at GeoParquet meetup yesterday, gdal properly filters on nested fields, but does not skip whole row groups.

That depends on the circumstances... In the general case indeed not, but a special case to handle that has been added in GDAL 3.8.0 to use the bbox from Ouverture Maps to skip row groups: OSGeo/gdal@c2ffc70
For the general case (arbitrary WHERE filter), more manual work would be needed, or use arrow::dataset filtering capabilities.
But for the purpose of spatial filtering, almost everything is already in place on the reading side.

3 replies

jwass Dec 5, 2023
Collaborator Author

Thanks. In that case I updated gdal to "yes".

You raise a good point that the predicate pushdown definition for our concern should really just be whether the proposed bbox definition is supported for spatial filtering... not whether it works in general. I'll update that as well.

jorisvandenbossche Dec 18, 2023
Maintainer

Thanks @rouault for correcting me here! I wasn't aware of GDAL having a custom implementation here to support such filters, that's cool. That's only supported when reading a single file, and not when reading multiple files at once? (i.e. a partitioned dataset, for which GDAL is using Arrow Datasets then, and then it's using Arrow's filtering capabilities?)

rouault Dec 18, 2023

That's only supported when reading a single file, and not when reading multiple files at once? (i.e. a partitioned dataset, for which GDAL is using Arrow Datasets then, and then it's using Arrow's filtering capabilities?)

yes, that's only for single files. Currently filters on ArrowDataset are only available on OGR side, without any use of statistics

jwass · 2023-12-05T21:26:10Z

jwass
Dec 5, 2023
Collaborator Author

As I mentioned in yesterday's meeting I made a dataset that might be useful for testing here. I put it up on source.coop https://beta.source.coop/repositories/jwasserman/geoparquet-spatial-query-testing/description/.

It is the November Overture Places releases, split into spatially separated files, with a bounding box column in both the nested structure format as well as top-level columns. A few more details on the source.coop page.

Here's a nice proof of duckdb's lack of support for predicate pushdown in nested columns - which we already knew but is nice to see with real results on a real-world dataset. These are two identical remote queries for a small region near the Boston area. The first queries the nested bbox fields, the second queries the top-level columns. The query and the data are otherwise identical. I ran using explain analyze to show the query plan and the execution results - notice how the second query plan has the filter during the read step.

The results for row-group filtering vs. non - just by switching from bbox.minx/... to minx...:

18x faster (6.2s vs 113.3s)
Transfers ~87x less data (23MB vs. 2GB)

Try it yourself. I'm hoping this dataset is useful for testing the other query engines and seeing how they perform reading the struct or the top-level columns.

┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││    Query Profiling Information    ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
explain analyze select count(*) from read_parquet('s3://us-west-2.opendata.source.coop/jwasserman/geoparquet-spatial-query-testing/overture_2023_11_14_alpha_0_places_sorted/*.parquet') where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││            HTTP Stats:            ││
││                                   ││
││             in: 2.0GB             ││
││            out: 0 bytes           ││
││             #HEAD: 83             ││
││             #GET: 795             ││
││              #PUT: 0              ││
││              #POST: 0             ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││        Total Time: 113.36s        ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌───────────────────────────┐
│      EXPLAIN_ANALYZE      │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             0             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│    UNGROUPED_AGGREGATE    │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│        count_star()       │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             1             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│  ((struct_extract(bbox,   │
│ 'minx') > -71.166345) AND │
│(struct_extract(bbox, ...  │
│       42.325606) AND      │
│(struct_extract(bbox, ...  │
│.050209) AND (struct_e...  │
│(bbox, 'maxy') < 42.40206))│
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│         EC: 20160         │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│           27180           │
│          (0.63s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│       READ_PARQUET        │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│            bbox           │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│         EC: 20160         │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│          57512517         │
│         (1688.99s)        │
└───────────────────────────┘
D

┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││    Query Profiling Information    ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
explain analyze select count(*) from read_parquet('s3://us-west-2.opendata.source.coop/jwasserman/geoparquet-spatial-query-testing/overture_2023_11_14_alpha_0_places_sorted/*.parquet') where minx > -71.166345 and miny > 42.325606 and maxx < -71.050209 and maxy < 42.40206;
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││            HTTP Stats:            ││
││                                   ││
││             in: 22.9MB            ││
││            out: 0 bytes           ││
││             #HEAD: 83             ││
││             #GET: 171             ││
││              #PUT: 0              ││
││              #POST: 0             ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││         Total Time: 6.21s         ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌───────────────────────────┐
│      EXPLAIN_ANALYZE      │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             0             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│    UNGROUPED_AGGREGATE    │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│        count_star()       │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             1             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│       READ_PARQUET        │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│  Filters: minx>-71.166345 │
│    AND minx IS NOT NULL   │
│ miny>42.325606 AND miny IS│
│          NOT NULL         │
│maxx<-71.050209 AND maxx IS│
│          NOT NULL         │
│ maxy<42.40206 AND maxy IS │
│          NOT NULL         │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│          EC: 4032         │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│           27180           │
│          (84.00s)         │
└───────────────────────────┘

1 reply

jwass Feb 13, 2024
Collaborator Author

I just reran the first query on the bbox struct using the latest duckdb version 0.10.0 and it now has identical performance!

D explain analyze select count(*) from read_parquet('s3://us-west-2.opendata.source.coop/jwasserman/geoparquet-spatial-query-testing/overture_2023_11_14_alpha_0_places_sorted/*.parquet') where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;
100% ▕████████████████████████████████████████████████████████████▏
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││    Query Profiling Information    ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
explain analyze select count(*) from read_parquet('s3://us-west-2.opendata.source.coop/jwasserman/geoparquet-spatial-query-testing/overture_2023_11_14_alpha_0_places_sorted/*.parquet') where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││            HTTP Stats:            ││
││                                   ││
││            in: 21.9 MiB           ││
││            out: 0 bytes           ││
││             #HEAD: 84             ││
││             #GET: 174             ││
││              #PUT: 0              ││
││              #POST: 0             ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌─────────────────────────────────────┐
│┌───────────────────────────────────┐│
││         Total Time: 3.03s         ││
│└───────────────────────────────────┘│
└─────────────────────────────────────┘
┌───────────────────────────┐
│      RESULT_COLLECTOR     │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             0             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│      EXPLAIN_ANALYZE      │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             0             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│    UNGROUPED_AGGREGATE    │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│        count_star()       │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│             1             │
│          (0.00s)          │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│       READ_PARQUET        │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│   Filters: bbox.minx>-71  │
│.166345 AND bbox.minx ...  │
│ NULL AND bbox.miny>42...  │
│  AND bbox.miny IS NOT NULL│
│ AND bbox.maxx<-71.050...  │
│  bbox.maxx IS NOT NULL AND│
│ bbox.maxy<42.40206 AN...  │
│     .maxy IS NOT NULL     │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│          EC: 4032         │
│   ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─   │
│           27180           │
│          (43.03s)         │
└───────────────────────────┘

Thanks, @Maxxen!

jwass · 2023-12-06T03:58:26Z

jwass
Dec 6, 2023
Collaborator Author

I'll look for more explicit verificationon Athena, but it looks like it is supported.

I'm not an expert on interpreting these query plans but it scans about 29MB which is a similar range to the duckdb pushdown-supported query and far less than the 2GB contained in a full scan of the bbox columns.

explain analyze select count(1) from overture_2023_11_14_alpha_0_places_sorted where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;

Query Plan
Queued: 256.52us, Analysis: 192.79ms, Planning: 835.18ms, Execution: 4.88s
Fragment 1 [SINGLE]
    CPU: 25.80ms, Scheduled: 26.53ms, Blocked 18.86s (Input: 15.10s, Output: 0.00ns), Input: 312 rows (2.74kB), Data Scanned: 0B; per task: avg.: 312.00 std.dev.: 0.00, Output: 1 row (9B)
    Output layout: [count]
    Output partitioning: SINGLE []
    Aggregate[type = FINAL]
    │   Layout: [count:bigint]
    │   Estimates: {rows: 1 (9B), cpu: 9, memory: 9B, network: 0B}
    │   CPU: 5.00ms (0.05%), Scheduled: 5.00ms (0.00%), Blocked: 0.00ns (0.00%), Output: 1 row (9B)
    │   Input avg.: 312.00 rows, Input std.dev.: 0.00%
    │   count := count("count_12")
    └─ LocalExchange[partitioning = SINGLE]
       │   Layout: [count_12:bigint]
       │   Estimates: {rows: 1 (9B), cpu: 0, memory: 0B, network: 0B}
       │   CPU: 8.00ms (0.09%), Scheduled: 8.00ms (0.00%), Blocked: 3.77s (19.98%), Output: 312 rows (2.74kB)
       │   Input avg.: 78.00 rows, Input std.dev.: 5.44%
       └─ RemoteSource[sourceFragmentIds = [2]]
              Layout: [count_12:bigint]
              CPU: 8.00ms (0.09%), Scheduled: 8.00ms (0.00%), Blocked: 15.10s (80.02%), Output: 312 rows (2.74kB)
              Input avg.: 78.00 rows, Input std.dev.: 5.44%

Fragment 2 [SOURCE]
    CPU: 9.21s, Scheduled: 4.08m, Blocked 0.00ns (Input: 0.00ns, Output: 0.00ns), Input: 500000 rows (9.58MB), Data Scanned: 29.24MB; per task: avg.: 16666.67 std.dev.: 63683.24, Output: 312 rows (2.74kB)
    Amount of input data processed by the workers for this stage might be skewed
    Output layout: [count_12]
    Output partitioning: SINGLE []
    Aggregate[type = PARTIAL]
    │   Layout: [count_12:bigint]
    │   Estimates: {rows: 1 (9B), cpu: 0, memory: 9B, network: 0B}
    │   CPU: 38.00ms (0.41%), Scheduled: 42.00ms (0.02%), Blocked: 0.00ns (0.00%), Output: 312 rows (2.74kB)
    │   Input avg.: 87.12 rows, Input std.dev.: 1763.52%
    │   count_12 := count(*)
    └─ ScanFilterProject[table = awsdatacatalog:default:overture_2023_11_14_alpha_0_places_sorted, filterPredicate = (("bbox#minx" > -7.1166345E1) AND ("bbox#miny" > 4.2325606E1) AND ("bbox#maxx" < -7.1050209E1) AND ("bbox#maxy" < 4.240206E1)), projectLocality = LOCAL, protectedBarrier = NONE]
           Layout: []
           Estimates: {rows: ? (0B), cpu: ?, memory: 0B, network: 0B}/{rows: ? (0B), cpu: ?, memory: 0B, network: 0B}/{rows: ? (0B), cpu: 0, memory: 0B, network: 0B}
           CPU: 9.16s (99.36%), Scheduled: 4.08m (99.97%), Blocked: 0.00ns (0.00%), Output: 27180 rows (0B)
           Input avg.: 1602.56 rows, Input std.dev.: 1269.80%
           bbox#maxy := bbox#maxy:double:REGULAR
           bbox#maxx := bbox#maxx:double:REGULAR
           bbox#miny := bbox#miny:double:REGULAR
           bbox#minx := bbox#minx:double:REGULAR

0 replies

jiayuasu · 2023-12-06T15:12:39Z

jiayuasu
Dec 6, 2023

@jwass We confirmed that Spark/Databricks parquet reader is able to perform filter pushdown on nested columns. Sedona geoparquet reader currently cannot perform filter pushdown nested columns. We will fix it in the next release. CC @Kontinuation

4 replies

jwass Dec 6, 2023
Collaborator Author

Great. Thanks for the info! I updated the first section.

Is there a PR or other link I can put there for us to track? Any idea when the next release will go out?

jiayuasu Dec 11, 2023

@jwass We created a JIRA ticket for this (https://issues.apache.org/jira/browse/SEDONA-440). The next release will be out in late December / early January, with other Sedona geoparquet writer issues: https://issues.apache.org/jira/browse/SEDONA-429 , https://issues.apache.org/jira/browse/SEDONA-430

jwass Jan 18, 2024
Collaborator Author

@jiayuasu Looks like these are all resolved and included in Sedona 1.5.1 - based on my reading of the release notes. Does Sedona now support predicate pushdown for the bbox grouped field?

jiayuasu Jan 19, 2024

@jwass All issues have been resolved in Sedona 1.5.1 and 1.5.1 has been released.

Sedona GeoParquet reader supports predicate pushdown for the bbox grouped field.

jatorre · 2024-01-16T14:13:26Z

jatorre
Jan 16, 2024

Just checked on BigQuery and it seems predicate push down work on this context

select count(1),1 as cachebuster from cartobq.maps.overtureplaces 

--When using this filter it has read 1.7GB of data from GCS
where bbox.minx > -180 and bbox.miny > -90 and bbox.maxx < 180 and bbox.maxy < 90

--When using this filter it has read 15MB
where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;

Being cartobq.maps.overtureplaces a VIRTUAL reference to the Geoparuqte files here gs://overturemapscarto/data_*.parquet

3 replies

jatorre Jan 16, 2024

And for example this query

select * EXCEPT(geometry),ST_GEOGFROMWKB(geometry) as geo from cartobq.maps.overtureplaces 
where bbox.minx > -71.166345 and bbox.miny > 42.325606 and bbox.maxx < -71.050209 and bbox.maxy < 42.40206;

Processes 97.42 MB and took 6 seconds

The parssing of the WKB took around 2seconds in an initial check.

jwass Jan 18, 2024
Collaborator Author

Awesome. Thanks @jatorre. I updated the summary discussion at the top.

jwass Jan 18, 2024
Collaborator Author

Were the files you were using (cartobq.maps.overtureplaces) the data I posted on source.coop (from #192 (comment))?

jatorre · 2024-01-18T14:16:03Z

jatorre
Jan 18, 2024

Yes, I just copied to Google cloud storage.

…

On Thu, Jan 18, 2024 at 3:05 PM Jacob Wasserman ***@***.***> wrote: Were the files you were using (cartobq.maps.overtureplaces) the data I posted on source.coop (from #192 (comment) <#192 (comment)> )? — Reply to this email directly, view it on GitHub <#192 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA7GOYAI5QVNDTH4ZALSSTYPET3NAVCNFSM6AAAAABAHZINHOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DCNRZG42DA> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

0 replies

jiayuasu · 2024-01-19T07:17:12Z

jiayuasu
Jan 19, 2024

@jwass

According to my communication with Snowflake team, Snowflake supports filter pushdown on grouped columns of parquet files.

But there is partial support of this feature in Snowflake. It has the following limitations:

It’s only applicable for primitive types, and only if all intermediate nested fields are structs (i.e. if there is an array or map in any level of nesting, it won’t work)
All expressions need to be casted extracts, even expressions not in the predicate. The filter pushdown is only supported on queries on vectorized scans. e.g. value:c1::int. Snowflake suggests this notation in our case: t.value:bbox:minX::int > 1

1 reply

jwass Jan 19, 2024
Collaborator Author

Thanks, @jiayuasu.

If we constrain this to just be about the current bbox proposal, it seems like the limitations would not apply and therefore predicate pushdown would be supported since it's primitive doubles with no intermediate nesting. Sounds like users will just have to do the casting t.value:bbox:minX::double > 1 but it should otherwise be supported?

Any chance you can run a test on the source.coop data from #192 (comment)

csringhofer · 2024-01-30T14:29:50Z

csringhofer
Jan 30, 2024

Fyi Apache Impala supports using Parquet row group and page level min/max stats for predicates on struct members.

set explain_level=2;
explain select * from bboxtbl where bbox.xmin > 0;
...
00:SCAN HDFS [functional_parquet.bboxtbl, RANDOM]
...
parquet statistics predicates: bbox.xmin > CAST(0 AS DOUBLE)
....

One issue though is that at the moment it cannot write Parquet files with complex types, so using separate double columns would be more convenient.

2 replies

jwass Jan 31, 2024
Collaborator Author

Thanks. Do you have any sense of whether it's a lot of work for Impala to support writing complex types? When we kicked this off we found a handful of tools didn't do row group filtering on nested fields on read, but they were able to add support relatively quickly.

csringhofer Feb 1, 2024

Adding some hack to be able to write bboxes wouldn't be that hard, but adding general support for writing complex types (or even just structs) is more problematic. The biggest issue is that Impala's UDF interface doesn't support complex types, so we cannot create a function at the moment that would return a struct, e.g. named_struct("xmin", 0, "ymin", 1 ... ) or get_bbox_as_struct(geom). Good question whether other engines/libraries have issues with complex type writing, or this an Impala only issue. It also seems possible that some engines can write structs, but do it much less efficiently than writing scalar columns.

I have read through #188 and have some question related to integration with SQL engines (not sure if this is the correct place to ask these).

table schema + writing:
Should the table's schema include the bbox column, or the engines should handle it as "hidden" index column that is not visible to the user? From INSERT point of view, should the INSERT statement include an extra column for the bbox, or the SQL engine should deduce that the extra column is needed and write it transparently?
While this is probably out of scope (as GeoParquet is a file format and not a table format), it would be nice to have a complete SQL example with CREATE + INSERT + SELECT on how this could work.
null handling + schema evolution:

SELECT * FROM parquet_backed_table WHERE bbox.xmin > -71.

On caveat is that "WHERE bbox.xmin > -71" will drop the row if bbox or bbox.xmin is NULL. This could be an issue if some files have a bbox column while some others don't, for example if this column was added later in the schema, so earlier files don't have it - some database engines replace missing columns with NULLs so will drop all rows in files without bbox column.
To handle NULLs something like WHERE coalesce(bbox.xmin, -infinity) > -71 is needed which is harder the "push down" than a simple > predicate.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Survey of Parquet Reader Support for Predicate Pushdown of Nested Bbox Definition #192

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 14 replies

{{title}}

gdal - No (@jorisvandenbossche)

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Survey of Parquet Reader Support for Predicate Pushdown of Nested Bbox Definition #192

jwass Dec 5, 2023 Collaborator

duckdb - Yes (as of 0.10.0) (@Maxxen)

gdal - Yes as of 3.8.0 (@rouault)

arrow-cpp - Yes as of 15.0 (@jorisvandenbossche)

Presto / Trino / Athena - Yes (@jwasserman)

Spark / Databricks - Yes (@jiayuasu)

Sedona - Yes as of 1.5.1 (@jiayuasu)

Snowflake Yes (@jiayuasu)

BigQuery - Yes (@jatorre)

Impala - Yes (@csringhofer)

Synapse ??

DataFusion - No (@kylebarron)

cudf - No (@paleolimbot)

Replies: 8 comments · 14 replies

gdal - No (@jorisvandenbossche)

jwass Dec 5, 2023 Collaborator Author

jorisvandenbossche Dec 18, 2023 Maintainer

jwass Dec 5, 2023 Collaborator Author

jwass Feb 13, 2024 Collaborator Author

jwass Dec 6, 2023 Collaborator Author

jwass Dec 6, 2023 Collaborator Author

jwass Jan 18, 2024 Collaborator Author

jwass Jan 18, 2024 Collaborator Author

jwass Jan 18, 2024 Collaborator Author

jwass Jan 19, 2024 Collaborator Author

jwass Jan 31, 2024 Collaborator Author

jwass
Dec 5, 2023
Collaborator

Replies: 8 comments 14 replies

jwass Dec 5, 2023
Collaborator Author

jorisvandenbossche Dec 18, 2023
Maintainer

jwass
Dec 5, 2023
Collaborator Author

jwass Feb 13, 2024
Collaborator Author

jwass
Dec 6, 2023
Collaborator Author

jwass Dec 6, 2023
Collaborator Author

jwass Jan 18, 2024
Collaborator Author

jwass Jan 18, 2024
Collaborator Author

jwass Jan 18, 2024
Collaborator Author

jwass Jan 19, 2024
Collaborator Author

jwass Jan 31, 2024
Collaborator Author