No JFileChooser Documents/Desktop/Downloads access prompt in macOS 10.15 Catalina

[sorghumking]

I'm submitting a…

• [] bug report
• [] feature request
• other

Short description of the issue/suggestion:
First, thank you for sharing your universalJavaApplicationStub. It enabled us to get Corelyzer running against Java 8+ on Mac.

Since using the stub, however, we've noticed an issue unique to macOS 10.15 Catalina. Bundled apps with the stub cannot access the Documents, Desktop, or Downloads folders through the JFileChooser. Unlike other apps, the macOS system prompt "[app] would like to access files in your Documents folder" introduced in macOS 10.15 Catalina does not automatically appear.

I'm not certain this is due to the stub, but I only see the behavior in bundled apps that use it. If it is not the stub, any guidance you can provide would be greatly appreciated.

I should also note that those dirs are accessible (without prompts) through FileDialog, which appears to use the native macOS chooser. That said, I'd like to understand and resolve the JFileChooser issue, if possible.

Steps to reproduce the issue/enhancement:
Build and run the sample app demonstrating the problem here:
https://github.com/sorghumking/catalinaFileChooserAccessTest

Or download and run the pre-built app here:
https://github.com/sorghumking/catalinaFileChooserAccessTest/releases

What is the expected behavior?
When attempting to access contents of the Documents, Desktop, or Downloads dir in JFileChooser, macOS should prompt "[app] would like to access files in your [folder name] folder".

What is the current behavior?
No prompt appears. The contents of the Documents, Desktop, and Downloads folders are not displayed.

Do you have screenshots, GIFs, demos or samples which demonstrate the problem or enhancement?
I've created a minimal sample app demonstrating the problem here:
https://github.com/sorghumking/catalinaFileChooserAccessTest

Screengrab:

What is the motivation / use case for changing the behavior?
Again, am I uncertain universalJavaApplicationStub is the culprit, but it's the only common thread I can find in apps that exhibit this problem...your thoughts would be much appreciated.

Please tell us about your environment:

• universalJavaApplicationStub version: v3.0.6 (2020-03-19)
• Mac OS version: 10.15.4 (Catalina)
• Java version(s): 8 (also observed this problem in Java 11)

[sorghumking]

EDIT: The Apple forum link is dead in the wake of Apple's recent WWDC forum revamp.

Thanks to helpful Apple forum folk, I've learned that this behavior is the result of using a script as an app's CFBundleExecutable. If I swap in a compiled binary for the CFBundleExecutable, access prompts ("[app] would like to access files in your Documents folder") appear as expected in JFileChooser.

I'm guessing there isn't a way to translate the universalJavaApplicationStub script into a compiled binary? :-)

[DerekCook]

Hi.

I am having the same problem. I don't really want to move away from JFileChooser as that will be a lot of work across several apps, and from I can see file filtering does not work well in FileDialog across platforms. But a quick hack to use FileDialog as a test in one of my apps verifies that FileDoalog does show files instead of me getting a blank list in JFileChooser.

Interestingly, even through the JFileChooser dialog is blank, I can actually load files which are under directory and are in my applications file history.

Is there a solution for this?

JFileChooser works when I run the application inside Eclipse as an EclipseLaunch configuration

This is on OS X Catalina for me and AdoptOpenJDK/JRE 11

[mbtaylor]

This isn't a fix, but it's an interesting observation: I develop an application which gives the user the option to use either java.awt.FileDialog or javax.swing.JFileChooser according to taste. I've seen the same; users can see these directories if they use FileDialog, but not if they use JFileChooser. However, I've had reports that if you use FileDialog to look at the offending directories, then subsequent uses of JFileChooser from the same application can see them too, "at least within that session, although it does remember it across sessions too, after a while".

I can't verify this personally, since I don't have Macs to play with.

[DerekCook]

Hi, I have just tried that, as my other dialogs in the application were still based on JFileChooser. So I opened the app, used the hacked file open first (FileChooser)m and then tried another file function (JFileChooser), and that does not seem to be the case.

[sorghumking]

...I've had reports that if you use FileDialog to look at the offending directories, then subsequent uses of JFileChooser from the same application can see them too, "at least within that session, although it does remember it across sessions too, after a while".

In the minimal app I built to demonstrate the problem:

https://github.com/sorghumking/catalinaFileChooserAccessTest/releases

I don't see the behavior you describe. Files in e.g. Documents are accessible in FileDialog, but remain inaccessible in JFileChooser after viewing in FileDialog. In the case of my app, the "[app] would like to access files in your Documents folder" prompt never appears. This is apparently a consequence of using a script (universalJavaApplicationStub) to launch the app.

I'd guess that if the prompt appears and access is granted, files would become accessible in JFileChooser as well.

[sorghumking]

Is there a solution for this?

JFileChooser works when I run the application inside Eclipse as an EclipseLaunch configuration

I also had no problems when running the app from VSCode - they only occur when running a proper app bundle.

I resolved the issue in my app by using the binary stub from this fork of Oracle's AppBundler:
https://github.com/TheInfiniteKind/appbundler
EDIT: I actually pulled the pre-built JAR from this repo: https://github.com/rlabduke/javadev/tree/master/lib/appbundler

AFAICT, there is no way to resolve the issue when the app is launched with a script like universalJavaApplicationStub.

[mbtaylor]

OK, I admit the FileDialog-fixes-JFileChooser thing sounded quite surprising to me. As I say, I can't vouch for it personally. There may be something strange about either my application or the way it's being used. If anybody wants to check it out, the full report was:

Actually, I do have a workaround - instead of using Filestore Browser, if I need to open something in the Documents or Downloads folders, System browser seems to be able to reach them. And, after opening a file directly from these folders (not one of the subfolders inside them), then Filestore Browser realizes it can access them as well (at least within that session, although it does remember it across sessions too, after a while).

and the bundled application is at http://www.starlink.ac.uk/topcat/topcat-full.dmg, but maybe the report was wrong.

[DerekCook]

Is there a solution for this?
JFileChooser works when I run the application inside Eclipse as an EclipseLaunch configuration

I also had no problems when running the app from VSCode - they only occur when running a proper app bundle.

I resolved the issue in my app by using the binary stub from this fork of Oracle's AppBundler:
https://github.com/TheInfiniteKind/appbundler

AFAICT, there is no way to resolve the issue when the app is launched with a script like universalJavaApplicationStub.

Thanks, I was actually thinking along the same lines for now, as that is how I originally got my application launcher for JRE7 until it stopped working under later OS X versions and I moved to universalJavaApplicationStub.

[tofi86]

Thanks for reporting this issue and sorry for the delay in answering...

I can partly reproduce this on 10.15.5 with your test app using Java 10 (will test other Java versions later on...).
I can access $HOME/Desktop without a problem with the JFileChooser, but Documents/ and Downloads/ indeed fail... Just showing a blank screen.

Long long time ago I moved away from JFileChooser in favor of FileDialog in all my applications because of the more "native" feeling of the FileDialog. That's maybe why I never stumbled upon this issue before...

First I thought you could work around this by setting one of the new Security Plist Keys for macOS 10.15 for Pretected Resources Access (https://developer.apple.com/documentation/bundleresources/information_property_list/protected_resources) – like [NSDownloadsFolderUsageDescription](https://developer.apple.com/documentation/bundleresources/information_property_list/nsdownloadsfolderusagedescription) but after reading the docs more thoroughly I guess these keys are really only for the message and not the access itself.

I tried anyways with

    <key>NSDownloadsFolderUsageDescription</key>
    <string>Download Access</string>

but neither of the dialogs showed the new Catalina security dialog like this:

Thanks to helpful Apple forum folk, I've learned that this behavior is the result of using a script as an app's CFBundleExecutable.

Unfortunately I cannot access the linked Apple developer forum thread for further research.

I'm guessing there isn't a way to translate the universalJavaApplicationStub script into a compiled binary? :-)

Haven't heard of a way to compile bash scripts yet :-(

That said, I did a quick research (https://lmgtfy.com/?q=compile+a+bash+script ;-) ) and clicked the first link https://www.simplified.guide/bash/compile-script, read about shc utility to compile bash to binary, installed with brew install shc and tried it with your test app:

The result is a Mach-O 64-bit executable x86_64 binary and HEY, IT WORKS! 🤩

So I may consider releasing binaries from now on on my releases page...

I added an issue for this (#87) to not forget the task...

[DerekCook]

Cool. I will try the same myself to see if it resolves the issue. I have been looking at packing the JRE as a rather heavy handed workaround, but whilst I have it working, it raises its own issues.

I had thought about migrating away from JFileChooser, but that would also entail a lot of works and checks, and unless I was missing the obvious choosable file filters looked more painful on FileDialog.

Anyway if we can create an executable from the script that is the path of least resistance for me. :) I will try that as soon as I get time.

[tofi86]

Would you be able to test my test binary release? ;-)

universalJavaApplicationStub-3.0.6-darwin-x86_64.zip

This ZIP file contains the source and binary of version 3.0.6:

• bash source universalJavaApplicationStub
• compiled binary universalJavaApplicationStub.x

I compiled on macOS 10.14.6 with shc -r -f universalJavaApplicationStub.

These are the sha256 checksums of both files:

• 5a557286b7efe87b0e293d1ea9b8b76c74217349a851d8214f9bc056dfcfd95e universalJavaApplicationStub
• a577295bf413fd0559cb403103ea393d35c252221a318e703d0afbbf5cee6aa7 universalJavaApplicationStub.x

You can verify them after download with openssl sha256 universalJavaApplicationStub.x.

[tofi86]

You would need to use the binary universalJavaApplicationStub.x in your App, of course. Just remove the .x extension after copying it to your app folder...

[DerekCook]

Sure thing. I will try it now....

[DerekCook]

Works perfectly.

The folder AN1x is under my Documents folder, and using the script I was just getting the blank directory just as the OP reported. With the compiled binary in place of the script inside my APP package then I now get this

Which enables this

Thanks for the prompt attention. It will be much quicker for me to adopt this as the solution as my path of least resistance.

[tofi86]

Awesome! 👏 Thanks for testing it so quickly! 🏅 💯

Glad it works for you...

I will officialy release binaries starting with the next 3.0.7 release. I'm also updating the README right now.

[DerekCook]

No problem. I am much happier to go with this as the solution. I have never embedded the JRE in my apps as I have eight of them with packages for MAC, Windows and Linux, so 24 uploads all around 25MB each, and I live in an area with appalling broadband speed (much better now I am using 4G broadband, but still not super fast), so ~600MB of uploads if I do a common release. Embedding the JRE would triple that amount.

On the plus side, I know how to embed the JRE if needed, but it is so much faff and the tools are still immature that I am much happier to stick with the launcher now you have a solution. So thanks for that.

[sorghumking]

This is excellent, I'm so glad a fairly straightforward solution was found.

(My apologies for the now-broken link to the Apple Developer forums where I received the initial clue regarding scripts vs. binaries. It appears my post got blown away in their most recent forum revamp.)

[tofi86]

Important note: From my latest experiments with shc I can report that compiling the launcher script on macOS 10.15 does not support running it on older macOS version like 10.14❗ That's pretty odd because for example GitHub Actions CI only offers Mac CI runners on 10.15.

Can someone test the binary above (#85 (comment)), which was compiled on 10.14, on macOS 10.13 or lower and report whether it's the same issue - app's not running?

[tofi86]

I opened an issue in the shc project: neurobin/shc#113

[amake]

Just FYI

I had all the issues here with OmegaT: script-based launcher, couldn't get the Catalina permissions prompts.

We ended up moving to https://github.com/TheInfiniteKind/appbundler, but in the meantime I discovered that granting /bin/sh (or whatever binary you're running the script with) Full Disk Access will let you work around the problem
(details here).

So when people report that "it works when I launch it from x", that means that x is a process owned by a binary that is able to request the permissions.

The idea of compiling our script never occurred to me; it's very clever, and if you can get it working with as broad compatibility as you're aiming for, then I will consider switching to this project in the future.

Thanks everyone for the great info in this thread.

[tofi86]

Binary releases are now published to the releases page of this repo starting with v3.1.0 (to be released in a couple of minutes)...

[tofi86]

Version 3.1.0 has officially been released 🎉

Details: https://github.com/tofi86/universalJavaApplicationStub/releases/tag/v3.1.0

[tofi86]

I'm also updating the README right now.

forgot that in the release. now fixed:

• Use a compiled binary for macOS 10.15 and above
• Access to "Protected Resources"

[davideby]

@tofi86 Sorry to comment on a closed issue, but have you tried using shc with the CFLAGS environment variable as mentioned in their man page? That allows setting the -mmacosx-version-minimum flag for shc to pass along to cc.

I'm using it to create a binary out of our launcher Bash script (which is admittedly much simpler than yours). Something like this:

$ export CFLAGS=-mmacosx-version-minimum=10.10
$ shc -r -f ./launcher.sh

It's probably excessive setting it for 10.10 (Yosemite) but figured I'd go with that since it's what our target JVM (AdoptOpenJDK) supports. I've tested on Sierra up through Big Sur with no problems so far, though.