Optimize Openstack Swift large file download

[MarekPikula]

Steps to reproduce

1. Set Openstack Swift as your primary storage (I'm using one from OVH on S1-8 server).
2. Upload large file (like 512 MB or so).
3. Download it (either desktop client or from web interface).

Expected behaviour

NextCloud server downloads file segments from object storage and simultaneously transfers them to client.

Actual behaviour

First entire file is downloaded to NextCloud server (OVH provides ~100Mb download link from their Object Storage from what I can see). Only after it's downloaded it's transferred to client. In the end it takes twice as much time.

I needed to change max_execution_time on server to prevent timeout errors.

Server configuration

Using Docker 15.0.4-php-alpine image.

Updated from an older Nextcloud/ownCloud or fresh install: Started at version 14.0

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:

• accessibility: 1.1.0
• activity: 2.8.2
• admin_audit: 1.5.0
• calendar: 1.6.4
• camerarawpreviews: 0.6.3
• checksum: 0.4.2
• cloud_federation_api: 0.1.0
• comments: 1.5.0
• contacts: 3.0.3
• data_request: 1.2.0
• dav: 1.8.1
• deck: 0.5.2
• drawio: 0.9.2
• encryption: 2.3.0
• event_update_notification: 0.3.2
• federatedfilesharing: 1.5.0
• federation: 1.5.0
• files: 1.10.0
• files_downloadactivity: 1.4.0
• files_external: 1.6.0
• files_markdown: 2.0.5
• files_mindmap: 0.0.11
• files_pdfviewer: 1.4.0
• files_rightclick: 0.11.0
• files_sharing: 1.7.0
• files_texteditor: 2.7.0
• files_trashbin: 1.5.0
• files_versions: 1.8.0
• files_videoplayer: 1.4.0
• firstrunwizard: 2.4.0
• flowupload: 0.0.9
• gallery: 18.2.0
• impersonate: 1.2.0
• logreader: 2.0.0
• lookup_server_connector: 1.3.0
• music: 0.9.3
• news: 13.0.3
• nextcloud_announcements: 1.4.0
• notes: 2.5.1
• notifications: 2.3.0
• oauth2: 1.3.0
• password_policy: 1.5.0
• previewgenerator: 2.0.0
• provisioning_api: 1.5.0
• quota_warning: 1.4.0
• ransomware_protection: 1.3.0
• richdocuments: 3.2.0
• scanner: 0.1.1
• serverinfo: 1.5.0
• sharebymail: 1.5.0
• spreed: 5.0.2
• support: 1.0.0
• survey_client: 1.3.0
• systemtags: 1.5.0
• tasks: 0.9.8
• theming: 1.6.0
• twofactor_backupcodes: 1.4.1
• twofactor_gateway: 0.11.0
• twofactor_nextcloud_notification: 1.0.3
• twofactor_totp: 2.1.0
• twofactor_u2f: 2.1.0
• twofactor_yubikey: 0.4.1
• unsplash: 1.1.3
• updatenotification: 1.5.0
• weather: 1.5.5
• workflowengine: 1.5.0
  Disabled:
• admin_notifications
• end_to_end_encryption
• files_accesscontrol
• files_fulltextsearch
• fulltextsearch
• twofactor_rcdevsopenotp
• user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "archive.cloud.pikula.co"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "15.0.4.0",
        "overwrite.cli.url": "https:\/\/new.cloud.pikula.co",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "objectstore": {
            "class": "OC\\Files\\ObjectStore\\Swift",
            "arguments": {
                "username": "dEcCjjVA5ghy",
                "password": "***REMOVED SENSITIVE VALUE***",
                "bucket": "cloud.pikula.co-data",
                "autocreate": false,
                "region": "WAW1",
                "url": "https:\/\/auth.cloud.ovh.net\/v2.0",
                "tenantName": "7435064151262691",
                "serviceName": "swift"
            }
        },
        "maintenance": false,
        "loglevel": 2,
        "theme": ""
    }
}

Are you using external storage, if yes which one: OpenStack Swift as primary storage

Are you using encryption: no

Are you using an external user-backend, if yes which one: No

Client configuration

Browser: Firefox 65.0 and NextCloud client 2.5.1final (build 20181204) AppImage

Operating system:

``` NAME=Fedora VERSION="29 (Server Edition)" ID=fedora VERSION_ID=29 VERSION_CODENAME="" PLATFORM_ID="platform:f29" PRETTY_NAME="Fedora 29 (Server Edition)" ANSI_COLOR="0;34" LOGO=fedora-logo-icon CPE_NAME="cpe:/o:fedoraproject:fedora:29" HOME_URL="https://fedoraproject.org/" DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f29/system-administrators-guide/" SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Fedora" REDHAT_BUGZILLA_PRODUCT_VERSION=29 REDHAT_SUPPORT_PRODUCT="Fedora" REDHAT_SUPPORT_PRODUCT_VERSION=29 PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy" VARIANT="Server Edition" VARIANT_ID=server ```

Logs

Well, in the end it works, but not in the optimal way. No errors are present (besides timeout, which was mitigated by changing timeout in php.ini).

[MarekPikula]

Do you need any more details to fix this bug, or there is simply nobody there to work on it?

[kirosbr]

Up, i have the same issue