Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overture Map Foundation's new release 2023-12-14-alpha.0 does not work #4

Open
PavlosDem99 opened this issue Jan 3, 2024 · 3 comments

Comments

@PavlosDem99
Copy link

I am trying to download OFM data using the new release 12-14 but I cout an error.
The error is:
[PATH_NOT_FOUND] Path does not exist: s3a://wherobots-public-data/overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=buildings/type=building.

image image

When i use the 07/26 release all are working perfectly.

Also, when I use this code from here Overture Map Data , also does not work.

from sedona.spark import *

config = SedonaContext.builder().config("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.AnonymousAWSCredentialsProvider").getOrCreate()
sedona = SedonaContext.create(config)

df = sedona.read.format("geoparquet").load("s3a://overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=places/type=place")
df.filter("ST_Contains(ST_GeomFromWKT('POLYGON((-122.48 47.43,-122.20 47.75,-121.92 47.37,-122.48 47.43))'), geometry) = true").show()

@jiayuasu
Copy link
Member

jiayuasu commented Jan 5, 2024

@PavlosDem99 Thanks for your interest. Our bucket only contains the first OMF release, the 07/26 release.

We will no longer provide new OMF data from our end because, with our help, OMF is now using Sedona to generate GeoParquet files. However, in OMF Nov and Dec releases, they didn't correctly partition data based on spatial proximity so the spatial filter is extremely slow. W are helping OMF fix this issue in the next release.

In fact, s3a://overturemaps-us-west-2/release/2023-12-14-alpha.0/theme=places/type=place works in our Sedona container and I just tried. But it is extremely slow because of the reason mentioned above.

@jiayuasu
Copy link
Member

jiayuasu commented Jan 5, 2024

@PavlosDem99 Took me about 5 min to download places data in Bellevue, WA

image

@PavlosDem99
Copy link
Author

@PavlosDem99 Took me about 5 min to download places data in Bellevue, WA

image

Thank you @jiayuasu for your reply I appreciate it a lot. You are doing amazing job you and your team!
Thanks I am waiting for the next release!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants