-
Notifications
You must be signed in to change notification settings - Fork 226
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JAR release to Maven Central #624
Comments
I'm not against it. I don't know how to accomplish it. If someone can provide a PR, if changes are necessary to source or build system, and clear instructions for what I need to do then this can be done. |
Does Khronos have a maven central account that is approved? The process is actually very easy it just looks daunting at first. |
I'll check. |
Khronos does not currently have an account. What is involved in syncing with the Maven repository? My colleague took a quick look, and it seems we can either host with Sonotype OSS repository hosting, or we can self host the repository and sync with Smart Proxy. https://central.sonatype.org/publish/large-orgs/sync/ What is the right approach here? |
You will probably want to setup the OSSRH to let them host it. The link you provided are the instructions. It is a bit clunky making a jira account to request a 'project'. Depending on how your organization is setup, you will want to request your highest level domain (no subdomain) for your packages. You can add the appropriate subdomain classpaths to each package you release. |
Turns out Khronos does have a Maven Central account and an established group id: org.khronos. We have now set up an account I can use for publishing the KTX jar files to this group id. Please look at pom.xml starting at the linked line. First there is stuff to do with signing and then, commented out, with publishing to Maven Central. Regarding signing, I couldn't spot where I'm supposed to provide the GPG signature. I also wonder if we can use our SSL.com-issued software signing certificate instead of a GPG signature. Regarding deployment, as well as fixing this up for our group id and figuring out how to provide the account name and pw to use, I need to know how to call this part of the POM from a CI deployment section. Can you make a PR with the necessary changes and clear indications of where I should provide signing key and account credentials? We do CI builds for Linux, macOS and Windows. Does it matter which one we use to deploy to Maven Central? The JAR files will all be the same right? |
@MarkCallow I'll need to look at how the build is setup. Normally for jars that are for jni bindings, the natives are packages in their own jar that makes the native available for java to bind to, as such, usually a separate jar would be produced for each platform. Here is an example from LWJGL: https://repo1.maven.org/maven2/org/lwjgl/lwjgl-vma/3.3.1/ As for the SSL.com, I'm not sure and will have to look into that. I have always used my own GPG signature when publishing to maven central. |
@MarkCallow It looks like you will need to generate and publish a GPG key. I do not see where SSL.com has services for GPG, but according to https://central.sonatype.org/publish/requirements/gpg/#distributing-your-public-key these are the only supported gpg hosts:
This (https://central.sonatype.org/publish/requirements/gpg/) has instructions on getting a GPG key setup and published. It will need to be available to the CI environment for the publishing, but the key itself should not be made public. |
For the case of KTX, one could/should consider to add the actual The DLLs cannot be loaded from the JAR files directly, though. They have to be written to a "temp"-directory to be loaded. And they have to be loaded in the "reverse order" of their dependency direction - meaning that it has to load Creating the JARs ... can be a bit tricky, too. In order to create the JARs, the native libraries have to be present. So building the natives and the JARs can hardly happen on the same machine, and even less in a lean-and-clean Signing the JARs will require a key, but this does not necessarily have to happen in the Maven build or CI. So... setting all this up properly can be a pain in the back. Depending on whether someone is willing to set up the signing as part of the official CI, an intermediate solution could be to create a build process that takes the precompiled natives, packs them into JARs, sign the JARs (manually), and upload them as 'Staging Bundles' in Sonatype (roughly as described in https://central.sonatype.org/publish/publish-manual/ ) |
I have no expertise in this. I would welcome a PR that sets up this publishing. To create a JAR files that contains the libs (dll/dylib/so) for each platform a manually triggered action similar to the one we use for publishing
What has to load the dlls, the application? What do you mean here by a "temp" directory? Note that on macOS dylibs have to be installed in the location matching their install-name attributes and libktx.dylib's location has to be listed in the RPATH attribute of libktx-jni.dylib. The install names for the dylibs included in the install package are |
Some context: My first goal would be to have a library that is not the KTX JAR itself, but a simpler, more Java-idiomatic convenience layer that is wrapped around that. But if that was supposed to be deployed to Maven, it would either
Sooo... paradoxically, it would safe me a ton of work if this issue was resolved, but ... I guess that I'll at least have to spend some time to look closer into this, and see whether I can contribute something to resolve it.
The DLLs have to be loaded "into the running JVM". On this level, it does not matter whether it is triggered by the application or by the (Java) library itself. But it does matter (a lot!), in terms of the user expecience. The current 'Usage' section at https://github.com/KhronosGroup/KTX-Software/tree/main/interface/java_binding#usage shows the example that includes the block static {
// Load libktx-jni, which provides the JNI stubs for natively
// implemented Java methods. This should also load libktx
// automatically! If it doesn't, you may need to load libktx manually.
System.loadLibrary("ktx-jni");
} So there, it is loaded by the application. But this So when someone wanted to deploy such an application, every user would have to manually twiddle with the (OS-specific) native libraries and put them into directories where they are "visible" for the JVM.
That's the common solution for this problem. It's unfortunate that this workaround is necessary, but ... it is. The point is: There is no function for loading a library from a
So when the DLL is deployed in a JAR via Maven, then there is only only feasible, user-transparent way to load it: (You can skip the links - these are only given as examples)
All this is surrounded by a load of boilerplate code for detecting the operating system and architecture. You'll have to differentiate between that You specifically pointed out an issue on MacOS (and I think that's your main development platform):
I don't know the technical backround here. What I do remember, though, is that I once stumbled over an issue in one of my projects (or a depencency of my projects) where that Related questions would be:
(There's always the option to say: "The Java bindings don't work on MacOS", depending on which hoops we can or cannot jump through here...) |
The KTX installers install both libraries into a location found on the PATH environment variable. (On Windows the install location is added to PATH.) So one user can install the libraries making them available to everyone else.
I think I understand. I'd be happy to receive a PR to add this stuff. On macOS binaries contain the paths to their dependent libraries which are set at link time. Those paths can be full paths or the path can use one of 3 prefixes that allow binaries to be installable anywhere:
The latter is constructed from the LC_RPATH commands in the binary. A library's install name can be a relative path or can use the @rpath prefix which permits dyld to match and load it for any application that has its location on its RPATH.
I'm not 100% sure. I think so, hence the @rpath stuff. Another issue is that on Apple Silicon all binaries must be signed and any attempts to modify an install name or rpath after signing will render the signature invalid.
Only one install name. Multiple LC_RPATH entries are allowed. I miswrote earlier. The install names for libktx.dylib and libktx-jni.dylib in our packages are What you want to do is possible. |
Poor @rpath 🙂 I can imagine your notifications. The point about the users having to handle the libraries may be a bit too focussed on the actual end-user experience. I'd like to correct the statement
to say "So one user has to install the libraries making them available to everyone else." (Imagine you downloaded an arbitrary application that could display, for example, glTF files. And during the installation it said: "You also have to install KTX-Software". You don't know what this is. You don't know what it does. You don't know why you need it. For an end-user, that's annoying...) How easy it is to make things really transparent and easy to use... is hard to say. But that |
I just read the dlopen man page. If the path passed to dlopen contains a slash then dlopen opens that file. If just a leaf name. |
Hello,
I saw some prior discussion on the matter, but would it be possible to get the jni binding jars released to maven central?
Maven central is the key location for libraries to be accessible via build tools.
Thanks,
Trevor
The text was updated successfully, but these errors were encountered: