S3 - Folder is seen as file with s3fs

[Loxeno]

Steps to reproduce

1. Mount a S3 bucket in Nextcloud with external_storage
2. Mount the same bucket using s3fs
3. With Nextcloud, create a folder 'test1', upload a file into it
4. With any other S3 client, create a folder 'test2', upload a file into it

Expected behaviour

ls -al

drwxrwxr-x 1 root root test1
drwxrwxr-x 1 root root test2

ls -al test1

drwxrwxr-x 1 root root file1

ls -al test2

drwxrwxr-x 1 root root file2

Actual behaviour

ls -al

-rwxrwxr-x 1 root root test1
drwxrwxr-x 1 root root test2

ls -al test1

ls -al test2

drwxrwxr-x 1 root root file2

I need to access these files though rexray/s3fs, many containers need to access the same data even if these apps do not use S3.

Server configuration

Operating system: Debian 9.8

Web server: Nginx 1.15.12

Database: MariaDB 10.3.15

PHP version: PHP 7.3.5

Nextcloud version: 16.0.1 using Docker latest official image

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: Docker

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - accessibility: 1.2.0
  - activity: 2.9.1
  - admin_audit: 1.6.0
  - apporder: 0.7.1
  - bookmarks: 1.0.5
  - bruteforcesettings: 1.3.0
  - calendar: 1.7.0
  - cloud_federation_api: 0.2.0
  - comments: 1.6.0
  - contacts: 3.1.1
  - dav: 1.9.2
  - deck: 0.6.2
  - drawio: 0.9.3
  - encryption: 2.4.0
  - event_update_notification: 0.3.4
  - external: 3.3.0
  - federatedfilesharing: 1.6.0
  - federation: 1.6.0
  - files: 1.11.0
  - files_external: 1.7.0
  - files_linkeditor: 1.0.10
  - files_markdown: 2.0.6
  - files_pdfviewer: 1.5.0
  - files_readmemd: 1.0.4
  - files_rightclick: 0.13.0
  - files_sharing: 1.8.0
  - files_texteditor: 2.8.0
  - files_trashbin: 1.6.0
  - files_versions: 1.9.0
  - files_videoplayer: 1.5.0
  - firstrunwizard: 2.5.0
  - gallery: 18.3.0
  - group_everyone: 0.1.2
  - groupfolders: 4.0.0
  - jsloader: 1.0.2
  - keeweb: 0.5.1
  - logreader: 2.1.0
  - lookup_server_connector: 1.4.0
  - news: 13.1.6
  - nextcloud_announcements: 1.5.0
  - notes: 2.6.0
  - notifications: 2.4.1
  - oauth2: 1.4.2
  - onlyoffice: 2.1.10
  - password_policy: 1.6.0
  - privacy: 1.0.0
  - provisioning_api: 1.6.0
  - registration: 0.4.6
  - serverinfo: 1.6.0
  - sharebymail: 1.6.0
  - sharelisting: 0.2.0
  - sharerenamer: 2.6.0
  - support: 1.0.0
  - survey_client: 1.4.0
  - systemtags: 1.6.0
  - tasks: 0.10.1
  - theming: 1.7.0
  - twofactor_backupcodes: 1.5.0
  - updatenotification: 1.6.0
  - viewer: 1.0.0
  - workflow_pdf_converter: 1.1.0
  - workflow_script: 1.1.0
  - workflowengine: 1.6.0
Disabled:
  - recommendations
  - twofactor_yubikey
  - user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "app.jonathan-ferme.fr"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "16.0.1.1",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "mail_sendmailmode": "smtp",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "loglevel": 2,
        "0": {
            "https": ""
        },
        "overwriteprotocol": "https",
        "app_install_overwrite": [
            "apporder",
            "keeweb",
            "twofactor_yubikey"
        ]
    }
}

Are you using external storage, if yes which one: S3
Are you using encryption: not with external storage
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log

No log even in debug.

Feel free to ask any further information.

By advance, thank you.

[kesselb]

Usually there are no empty folders with s3. Put a file into the folder and you should see the folder.

[Loxeno]

Even with a file, I get the same result.
And if I create an empty folder using another S3 client (in my case, Scaleway Object Storage explorer), I can see this object as a folder in s3fs.

That's why I think there is an issue here.

[kesselb]

Try this patch #14913

[Loxeno]

Yes, just find it, will test it tomorrow.

I'll tell you what happen then.

Thank you :)

[Loxeno]

Hi,

Thank you very much, your patch worked perfectly !

[Loxeno]

I said worked too quickly without properly check what happened with s3fs...

The patch solved one part of the problem, which is not being able to browse folders/files previously created with other S3 client (minio in my case) with NC.

Folders created by NC are still seen as file with s3fs...

I can't say if the problem is limited to NC or RexRay / s3fs.
I will open an issue to RexRay in order to get some help.

[Loxeno]

Hi,

Have you been able to reproduce it ?

[Loxeno]

Hi,

Is there any improvment ?

[kesselb]

s3fs-fuse/s3fs-fuse#885. Patch is part of s3fs 1.85.