Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SEDONA-177] Extend RDD API for RangeQuery JoinQuery to support spatial predicates other than coveredBy and intersects. #700

Merged
merged 5 commits into from
Oct 18, 2022

Conversation

Kontinuation
Copy link
Member

@Kontinuation Kontinuation commented Oct 14, 2022

Did you read the Contributor Guide?

Is this PR related to a JIRA ticket?

What changes were proposed in this PR?

We've extended the RDD APIs for range query and spatial/distance join query.

Spatial Predicate Enums

We'll define 9 commonly used spatial predicates in enum SpatialPredicate:

public enum SpatialPredicate {
    CONTAINS,
    INTERSECTS,
    WITHIN,
    COVERS,
    COVERED_BY,
    TOUCHES,
    OVERLAPS,
    CROSSES,
    EQUALS
}

Spatial Range Query

User can specify one of the spatial predicates in RangeQuery.SpatialRangeQuery.

val rangeQueryWindow = new Envelope(-90.01, -80.01, 30.01, 40.01)
val usingIndex = false
var queryResult = RangeQuery.SpatialRangeQuery(spatialRDD, rangeQueryWindow, SpatialPredicate.COVERED_BY, usingIndex)
// this is equivalent to
//    RangeQuery.SpatialRangeQuery(rangeQueryWindow, spatialRDD, SpatialPredicate.COVERS, usingIndex)
// or
//    val considerBoundaryIntersection = false
//    RangeQuery.SpatialRangeQuery(spatialRDD, rangeQueryWindow, considerBoundaryIntersection, usingIndex)

Spatial Join Query

User can specify one of the spatial predicates in JoinQuery.SpatialJoinQuery.

val usingIndex = false
var result = JoinQuery.SpatialJoinQuery(objectRDD, queryWindowRDD, usingIndex, SpatialPredicate.COVERED_BY)
// this is equivalent to
//    val considerBoundaryIntersection = false
//    JoinQuery.SpatialJoinQuery(objectRDD, queryWindowRDD, usingIndex, considerBoundaryIntersection)
var result = JoinQuery.SpatialJoinQuery(objectRDD, queryWindowRDD, usingIndex, SpatialPredicate.INTERSECTS)
// this is equivalent to
//    val considerBoundaryIntersection = true
//    JoinQuery.SpatialJoinQuery(objectRDD, queryWindowRDD, usingIndex, considerBoundaryIntersection)

Distance Join Query

JoinQuery.DistanceJoinQuery only accepts INTERSECTS or COVERED_BY as spatial predicates.

val usingIndex = false
var result = JoinQuery.DistanceJoinQuery(objectRDD, queryRDD, usingIndex, SpatialPredicate.COVERED_BY)
// this is equivalent to
//    val considerBoundaryIntersection = false
//    JoinQuery.DistanceJoinQuery(objectRDD, queryRDD, usingIndex, considerBoundaryIntersection)

Spatial join optimizer in Sedona SQL

Spatial join executors now correctly evaluate spatial predicates (ST_*), which resolves SEDONA-119.

How was this patch tested?

Added test cases for newly added APIs. The new data files containing test data were randomly generated.

Added test cases for SEDONA-119, which checks whether spatial predicates as spatial join conditions were correctly evaluated.

Did this PR include necessary documentation updates?

  • Yes, I have updated the documentation update.

@Kontinuation Kontinuation changed the title WIP: [SEDONA-177] Extend RDD API for RangeQuery JoinQuery to support spatial predicates other than coveredBy and intersects. [SEDONA-177] Extend RDD API for RangeQuery JoinQuery to support spatial predicates other than coveredBy and intersects. Oct 15, 2022
@jiayuasu jiayuasu added this to the sedona-1.3.0 milestone Oct 15, 2022
@jiayuasu
Copy link
Member

@Imbruced This great PR introduces a set of new Spatial Predicates (https://github.com/apache/incubator-sedona/pull/700/files#diff-1ed309b8e78014ee8b34da22e2a1f90894f1d99590924f85ac88b730962f21ce) to address some corner cases in Sedona range and join queries.

For the Python SQL part, this change should be transparent. But for the Python RDD part, we need to introduce a Python wrapper around it. Do you want to take care of this part?

@jiayuasu
Copy link
Member

I think this PR looks good to me. If Pawel wants to take care of the Python wrapper side in a follow-up PR, I can just merge this PR.

@jiayuasu jiayuasu merged commit b2ee308 into apache:master Oct 18, 2022
@Kontinuation Kontinuation deleted the correct-join-predicate branch August 23, 2023 15:01
@Kontinuation Kontinuation restored the correct-join-predicate branch August 23, 2023 15:01
@Kontinuation Kontinuation deleted the correct-join-predicate branch August 23, 2023 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants