Added deployment workflow for rust API

[benjaminwinger]

Note that this will publish the version as recorded in tools/rust_api/Cargo.toml, so that will need to be updated before the deployment pipeline is triggered.

Alternatively we could have a script which detects the version and rewrites the file to use the most up to date version, but it would be helpful to keep the version up to date so that anyone adding kuzu as a git dependency sees a meaningful version.

[mewim]

Since this is a very simple job that does not require compilation, consider move it to a GitHub-hosted runner instead of using our own infrastructure.

[benjaminwinger]

The choice was deliberate. It's running a full build of the crate before uploading it, which is particularly important since the rust API only includes certain files when packaging both because the tools/rust_api/kuzu-src symlink breaks the way it usually omits untracked git files, and because there's a maximum upload size and including stuff like the dataset directory would make the result too large. There's no verification that it can be built with the included files in CI, so having this last-minute check is important (and while it's . Having the check in CI instead would be slow, but would probably be a lot less so with sccache set up correctly.

[mewim]

Note that kuzu-self-hosted-linux-building is a different environment than the regular CI. It is based on CentOS 7 as defined in https://github.com/kuzudb/kuzu/blob/master/scripts/pip-package/Dockerfile. Could you please also update the Dockerfile to add rust dependencies? I will redeploy after this PR is merged.

[benjaminwinger]

Oh, that I missed. I confused it with linux-self-hosted-testing. Since the results of the build aren't what's being published, it probably should just use the testing environment shouldn't it? I don't know what exactly the difference between the two environments are but I'm assuming it's something to do with the libraries it links against.

[mewim]

Yes the building environment uses a distro with old glibc so the prebuilt binaries can have better compatibility. You can use the testing environment since the binaries are not going to be published.

[benjaminwinger]

I have updated it to use kuzu-self-hosted-testing

[mewim]

For cargo publish, is there a way to dry-run or deploy to a test environment similar to npm or pypi? If so, we should do dry run unless isDeploy is set to true so we can perform a test run without actually deploying.

[benjaminwinger]

Yes, there's a --dry-run option which skips uploading. I don't think there's a test registry like test.pypi.org, but we could just upload the packaged crate file to github.

[mewim]

OK. Please add the dry-run based on the flag. We should always upload the packaged files to GitHub as well for debugging purpose in case something goes wrong and the deployment fails.

[benjaminwinger]

Done.

[mewim]

Alternatively we could have a script which detects the version and rewrites the file to use the most up to date version.

Let's add the script and invoke it on CI to ensure that the deployment can still be done correctly even if we forget to update the version for rust. You can get the version number from CMakeLists.txt similar to Node.js and Python APIs.

[mewim]

It seems that the path here is not correct. I tested run this job once and it’s not able to find any artifact. https://github.com/kuzudb/kuzu/actions/runs/5522371969/jobs/10071856122

[benjaminwinger]

Whoops. I wrote it relative to tools/rust_api, but that's not the working directory for that step.

[codecov]

Codecov Report

Patch and project coverage have no change.

Comparison is base (7ec4007) 91.13% compared to head (0591fdd) 91.13%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1800   +/-   ##
=======================================
  Coverage   91.13%   91.13%           
=======================================
  Files         776      776           
  Lines       28373    28373           
=======================================
+ Hits        25857    25858    +1     
+ Misses       2516     2515    -1     

see 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.