Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-44065: [Java] Implement C Data Interface for RunEndEncodedVector #44241

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

ViggoC
Copy link
Contributor

@ViggoC ViggoC commented Sep 26, 2024

@vibhatha
Copy link
Collaborator

@ViggoC it would be better to add a test here as well https://github.com/apache/arrow/blob/main/java/c/src/test/python/integration_tests.py
Please refer to the equivalent once for StringView and ListView

@ViggoC
Copy link
Contributor Author

ViggoC commented Sep 26, 2024

@vibhatha Your response is really quickly. I am still trying to understand your PR #41967, as well as how to run tests. If you could give me some guidance, it would be of great help.
So just integration_tests.py is must to have? How to run it?

@vibhatha
Copy link
Collaborator

Right, so there are few things we have to do here.
First try to build the C data interface related code, you can get an idea here. This way you can run the test cases.

For integration tests, you need to make sure to use archery. Refer to this. Make sure archery is installed and all other relevant things. I would use a mamba environment for this. Please refer to the Python development guidelines to learn more about using conda/mamba for that. CLI help menu can give more guidance. Make sure to test both IPC, and C Data tests.

One more important thing is the ComplexWriter component which contains the JSON based writers and readers which are key factors for these tests.

If you have any questions, please ask here. And thanks for pushing this effort.

@ViggoC
Copy link
Contributor Author

ViggoC commented Oct 8, 2024

@vibhatha @lidavidm If I want to release this feature in version 18.0, when is the due date?

@vibhatha
Copy link
Collaborator

vibhatha commented Oct 8, 2024

cc @raulcd

@lidavidm
Copy link
Member

lidavidm commented Oct 8, 2024

@ViggoC ViggoC marked this pull request as ready for review October 8, 2024 14:29
@@ -770,6 +771,17 @@ public void testStructVector() {
}
}

@Test
public void testRunEndEncodedVector() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shall we also add a test for an empty REE Vector?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, see testEmptyRunEndEncodedVector

Copy link
Collaborator

@vibhatha vibhatha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only went through one quick pass, I will review again.

@github-actions github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Oct 9, 2024
@vibhatha
Copy link
Collaborator

vibhatha commented Oct 9, 2024

@github-actions crossbow submit -g java

Copy link

github-actions bot commented Oct 9, 2024

Revision: b34aca3

Submitted crossbow builds: ursacomputing/crossbow @ actions-302154c076

Task Status
java-jars GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants