Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Out of Memory error when uploading large file via public.php #12449

Closed
Toasterson opened this issue Nov 14, 2018 · 4 comments
Closed

Out of Memory error when uploading large file via public.php #12449

Toasterson opened this issue Nov 14, 2018 · 4 comments
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info

Comments

@Toasterson
Copy link

Steps to reproduce

  1. Create a Writeable Share Link and note it's URL
  2. Create a test file bash dd if=/dev/urandom of=testfile_512 bs=1M count=512
  3. Upload to https://nc.example.com/public.php/webdav/testfile_512 bash cat testfile_512 | curl -i -X PUT -u '${HASH_OF_SHARE_URL}:' --data-binary @/dev/stdin "https://nc.example.com/public.php/webdav/testfile_512"

Expected behaviour

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
etc. etc.

Actual behaviour

Out of Memory Error. Apparently public.php does not use the temporary upload directory? While remote.php does

HTTP/1.1 100 Continue

HTTP/1.1 500 Internal Server Error
etc. etc.

Nov 14 11:41:38 $SERVER Nextcloud [$USER][$PID]: {PHP} Allowed memory size of 1073741824 bytes exhausted (tried to allocate 536870913 bytes) at /path/to/nextcloud/lib/private/AppFramework/Http/Request.php#457

Server configuration

Operating system:
CentOS 7.5

Web server:
Apache httpd.x86_64 2.4.6-80.el7.centos.1

Database:
mariadb.x86_64 1:5.5.60-1.el7_5

PHP version:
php-cli.x86_64 7.1.23-1.el7.remi @remi-php71
php-common.x86_64 7.1.23-1.el7.remi @remi-php71
php-devel.x86_64 7.1.23-1.el7.remi @remi-php71
php-fedora-autoloader.noarch 1.0.0-1.el7.remi @remi
php-fpm.x86_64 7.1.23-1.el7.remi @remi-php71
php-gd.x86_64 7.1.23-1.el7.remi @remi-php71
php-intl.x86_64 7.1.23-1.el7.remi @remi-php71
php-json.x86_64 7.1.23-1.el7.remi @remi-php71
php-mbstring.x86_64 7.1.23-1.el7.remi @remi-php71
php-mcrypt.x86_64 7.1.23-1.el7.remi @remi-php71
php-mysqlnd.x86_64 7.1.23-1.el7.remi @remi-php71
php-opcache.x86_64 7.1.23-1.el7.remi @remi-php71
php-pdo.x86_64 7.1.23-1.el7.remi @remi-php71
php-pear.noarch 1:1.10.6-1.el7.remi @remi
php-pecl-apcu.x86_64 5.1.12-1.el7.remi.7.1 @remi-php71
php-pecl-apcu-devel.x86_64 5.1.12-1.el7.remi.7.1 @remi-php71
php-pecl-zip.x86_64 1.15.4-1.el7.remi.7.1 @remi-php71
php-process.x86_64 7.1.23-1.el7.remi @remi-php71
php-xml.x86_64 7.1.23-1.el7.remi @remi-php71

Nextcloud version: (see Nextcloud admin page)
14.0.0
14.0.3

Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 13.x

Where did you install Nextcloud from:
nextcloud.org archive tarball

Signing status:

Signing status 
No errors have been found.

List of activated apps:

App list 
Enabled:
  - accessibility: 1.0.1
  - cloud_federation_api: 0.0.1
  - comments: 1.4.0
  - dav: 1.6.0
  - federatedfilesharing: 1.4.0
  - files: 1.9.0
  - files_accesscontrol: 1.4.0
  - files_markdown: 2.0.4
  - files_pdfviewer: 1.3.2
  - files_sharing: 1.6.2
  - files_texteditor: 2.6.0
  - files_trashbin: 1.4.1
  - files_versions: 1.7.1
  - files_videoplayer: 1.3.0
  - firstrunwizard: 2.3.0
  - gallery: 18.1.0
  - lookup_server_connector: 1.2.0
  - notifications: 2.2.1
  - oauth2: 1.2.1
  - provisioning_api: 1.4.0
  - sharebymail: 1.4.0
  - support: 1.0.0
  - systemtags: 1.4.0
  - twofactor_backupcodes: 1.3.1
  - twofactor_totp: 1.5.0
  - twofactor_u2f: 1.6.1
  - workflowengine: 1.4.0
Disabled:
  - admin_audit

Nextcloud configuration:

Config report 
{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "3732213.stoney-storage.com",
            "tescht.eichenberger.ch"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "https:\/\/3732213.stoney-storage.com",
        "dbtype": "mysql",
        "version": "14.0.0.19",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "config_is_read_only": true,
        "appstoreenabled": false,
        "log_type": "syslog",
        "log_level": 2,
        "log_timezone": "UTC",
        "syslog_tag": "Nextcloud [3732213]",
        "updatechecker": false,
        "forcessl": true,
        "forceSSLforSubdomains": true,
        "theme": "stoney-storage",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "sendmail",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": 25,
        "mail_smtptimeout": 10,
        "mail_smtpsecure": "",
        "mail_smtpauth": false,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "loglevel": 2,
        "maintenance": false
    },
    "apps": {
        "accessibility": {
            "enabled": "yes",
            "installed_version": "1.0.1",
            "types": ""
        },
        "activity": {
            "enabled": "no",
            "installed_version": "2.5.2",
            "types": "filesystem"
        },
        "backgroundjob": {
            "lastjob": "24"
        },
        "cloud_federation_api": {
            "enabled": "yes",
            "installed_version": "0.0.1",
            "types": "filesystem"
        },
        "comments": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "logging"
        },
        "core": {
            "backgroundjobs_mode": "cron",
            "installedat": "1528360727.9279",
            "lastcron": "1542192970",
            "lastupdatedat": "0",
            "moveavatarsdone": "yes",
            "oc.integritycheck.checker": "[]",
            "previewsCleanedUp": "1",
            "public_files": "files_sharing\/public.php",
            "public_webdav": "dav\/appinfo\/v1\/publicwebdav.php",
            "scss.variables": "6aa77fbb45598b87bd3037cdb0e5d13d",
            "vendor": "nextcloud"
        },
        "dav": {
            "buildCalendarSearchIndex": "yes",
            "enabled": "yes",
            "installed_version": "1.6.0",
            "types": "filesystem"
        },
        "federatedfilesharing": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": ""
        },
        "federation": {
            "enabled": "no",
            "installed_version": "1.2.0",
            "types": "authentication"
        },
        "files": {
            "cronjob_scan_files": "500",
            "enabled": "yes",
            "installed_version": "1.9.0",
            "types": "filesystem"
        },
        "files_accesscontrol": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "filesystem"
        },
        "files_markdown": {
            "enabled": "yes",
            "installed_version": "2.0.4",
            "types": ""
        },
        "files_pdfviewer": {
            "enabled": "yes",
            "installed_version": "1.3.2",
            "types": ""
        },
        "files_sharing": {
            "enabled": "yes",
            "installed_version": "1.6.2",
            "types": "filesystem"
        },
        "files_texteditor": {
            "enabled": "yes",
            "installed_version": "2.6.0",
            "types": ""
        },
        "files_trashbin": {
            "enabled": "yes",
            "installed_version": "1.4.1",
            "types": "filesystem,dav"
        },
        "files_versions": {
            "enabled": "yes",
            "installed_version": "1.7.1",
            "types": "filesystem,dav"
        },
        "files_videoplayer": {
            "enabled": "yes",
            "installed_version": "1.3.0",
            "types": ""
        },
        "firstrunwizard": {
            "enabled": "yes",
            "installed_version": "2.3.0",
            "types": "logging"
        },
        "gallery": {
            "enabled": "yes",
            "installed_version": "18.1.0",
            "types": ""
        },
        "logreader": {
            "enabled": "no",
            "installed_version": "2.0.0",
            "ocsid": "170871",
            "types": ""
        },
        "lookup_server_connector": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "authentication"
        },
        "nextcloud_announcements": {
            "enabled": "no",
            "installed_version": "1.1",
            "types": "logging"
        },
        "notifications": {
            "enabled": "yes",
            "installed_version": "2.2.1",
            "types": "logging"
        },
        "oauth2": {
            "enabled": "yes",
            "installed_version": "1.2.1",
            "types": "authentication"
        },
        "password_policy": {
            "enabled": "no",
            "installed_version": "1.2.2",
            "types": ""
        },
        "provisioning_api": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "prevent_group_restriction"
        },
        "serverinfo": {
            "enabled": "no",
            "installed_version": "1.2.0",
            "types": ""
        },
        "sharebymail": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "filesystem"
        },
        "support": {
            "enabled": "yes",
            "installed_version": "1.0.0",
            "types": ""
        },
        "survey_client": {
            "enabled": "no",
            "installed_version": "1.0.0",
            "types": ""
        },
        "systemtags": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "logging"
        },
        "theming": {
            "enabled": "no",
            "installed_version": "1.3.0",
            "types": "logging"
        },
        "twofactor_backupcodes": {
            "enabled": "yes",
            "installed_version": "1.3.1",
            "types": ""
        },
        "twofactor_totp": {
            "enabled": "yes",
            "installed_version": "1.5.0",
            "types": ""
        },
        "twofactor_u2f": {
            "enabled": "yes",
            "installed_version": "1.6.1",
            "types": ""
        },
        "updatenotification": {
            "enabled": "no",
            "installed_version": "1.2.0",
            "types": ""
        },
        "workflowengine": {
            "enabled": "yes",
            "installed_version": "1.4.0",
            "types": "filesystem"
        }
    }
}

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Curl

Operating system: CentOS 7.5

Logs

Web server error log

Web server error log 
[Wed Nov 14 11:41:39.207129 2018] [proxy_fcgi:error] [pid 24394] [client 31.216.41.251:44264] AH01071: Got error 'PHP message: PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 536870913 bytes) in /path/to/nextcloud/lib/private/AppFramework/Http/Request.php on line 457\n'

Nextcloud log (data/nextcloud.log)

Nextcloud log 
Nov 14 11:41:38 $HOST Nextcloud [$USER][$PID]: {PHP} Allowed memory size of 1073741824 bytes exhausted (tried to allocate 536870913 bytes) at /path/to/nextcloud/lib/private/AppFramework/Http/Request.php#457

Browser log

Browser log 
HTTP/1.1 100 Continue

HTTP/1.1 500 Internal Server Error
Date: Wed, 14 Nov 2018 10:41:32 GMT
Server: Apache/2.4.6 (CentOS)
X-Powered-By: PHP/7.1.23
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-eval' 'nonce-NDQvSXB2cHNFd0tnUnl1blpaNnJkVVpkQXkveFRDRldrYmtkOUVScVF6VT06b3VTSjdiNExhMmp3QWtQcUovWEpSQ2NaWW5taUZXUmorTXQvaHdWY0VIRT0='; style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
X-Frame-Options: SAMEORIGIN
Set-Cookie: ocr3s489ab17=819802318f8360ae040a43b9dc7d57af; path=/; secure; HttpOnly
Set-Cookie: oc_sessionPassphrase=%2BfRNOUXUovUmvsw4vHuIe6gEUINRMsTZAbl7%2BIQnbbSsmV5Vgm6d9mVT2lurxlFEkktAIoh7B9II00qz%2FVZBKuDtZs85XhPt6Kqyga2udvK3Y60gb995md4JEU%2FNPR8I; path=/; secure; HttpOnly
Set-Cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
Set-Cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
Strict-Transport-Security: max-age=15768000
Referrer-Policy: strict-origin
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Robots-Tag: none
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
@nextcloud-bot
Copy link
Member

GitMate.io thinks possibly related issues are #6784 (Php error?), #9263 (Error when uploading files via web), #4392 (Error while syncing/uploading large files), #3296 (Error Message if uploading files), and #8954 (Uploading large files fails).

@loxK
Copy link

loxK commented Dec 4, 2018

I have the same issue, the memory used is insane, with a memory limit of 1.5G I cannot upload a 512Mo file

@kesselb kesselb added 0. Needs triage Pending check for reproducibility or if it fits our roadmap 14-feedback 1. to develop Accepted and waiting to be taken care of and removed 0. Needs triage Pending check for reproducibility or if it fits our roadmap labels May 2, 2019
@kesselb
Copy link
Contributor

kesselb commented May 2, 2019

I can reproduce this with Nextcloud 16. cc @nextcloud/server-triage

@MorrisJobke MorrisJobke added this to the Nextcloud 16.0.2 milestone May 7, 2019
@skjnldsv skjnldsv removed this from the Nextcloud 16.0.11 milestone Apr 27, 2020
@szaimen
Copy link
Contributor

szaimen commented Jan 9, 2023

Hi, please update to 24.0.8 or better 25.0.2 and report back if it fixes the issue. Thank you!

@szaimen szaimen added needs info 0. Needs triage Pending check for reproducibility or if it fits our roadmap and removed 1. to develop Accepted and waiting to be taken care of labels Jan 9, 2023
@szaimen szaimen closed this as completed Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
0. Needs triage Pending check for reproducibility or if it fits our roadmap bug needs info
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
9 participants
@rullzer @loxK @nickvergessen @MorrisJobke @Toasterson @kesselb @skjnldsv @nextcloud-bot @szaimen