Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SEDONA-287] Introduce SedonaContext to unify Sedona entry points #848

Merged
merged 5 commits into from
Jun 6, 2023

Conversation

jiayuasu
Copy link
Member

@jiayuasu jiayuasu commented Jun 3, 2023

Did you read the Contributor Guide?

Is this PR related to a JIRA ticket?

What changes were proposed in this PR?

  1. Introducing a new entry point called SedonaContext to unify all Sedona entry points in different compute engines and deprecate old Sedona register entry points. Users no longer have to register Sedona kryo serializer and import many tedious Python classes.
  2. Together with this PR, a new Spark Spark python dir called sedona.spark is also introduced. It imports all essential Sedona Python classes so the users can simply call from sedona.spark import * to import everything in one line of code.

Note that: this does not prevent any existing Sedona users from upgrading to the new version. We can remove the deprecated entry pointss after several major releases.

Sedona Spark

Scala

import org.apache.sedona.spark.SedonaContext

val sedona = SedonaContext.create(SedonaContext.builder().master("local[*]").getOrCreate())

sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM")

Python

from sedona.spark import *

config = SedonaContext.builder() .\
    config('spark.jars.packages',
           'org.apache.sedona:sedona-spark-shaded-3.0_2.12:1.4.1,'
           'org.datasyslab:geotools-wrapper:1.4.0-28.2'). \
    getOrCreate()

sedona = SedonaContext.create(config)

sedona.sql("SELECT ST_GeomFromWKT(XXX) FROM")

Sedona Flink

import org.apache.sedona.flink.SedonaContext

StreamTableEnvironment sedona = SedonaContext.create(env, tableEnv);

sedona.sqlQuery("SELECT ST_GeomFromWKT(XXX) FROM")

How was this patch tested?

Passed existing unit tests

Did this PR include necessary documentation updates?

@jiayuasu jiayuasu requested a review from Sarwat June 3, 2023 05:42
@jiayuasu jiayuasu merged commit d2dd743 into master Jun 6, 2023
39 checks passed
@jiayuasu jiayuasu deleted the sedona-context branch June 19, 2023 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants