expiry of a shared link leads to removal of remaining link with expiry in the future (multiple shared link feature)

[hmalessa]

Steps to reproduce

1. create/have one shared link with expiration date
2. create a second shared link (for same thing of course) with an expiration date after first link's expiry
3. wait until expiration date of first link

Expected behaviour

only the first link should be removed. The second link has to remain.

Actual behaviour

all links (for the same folder) are removed.
database table OC_SHARE has no more rows with share_type=3.

Server configuration

Operating system:
Ubuntu Server 18.04.1 LTS
Kernel 4.4.0-133 x86_64
Web server:
apache2 2.4.18-2ubuntu3.9
Database:
mysql Ver 14.14 Distrib 5.7.23
PHP version:
PHP 7.0.30-0ubuntu0.16.04.1
Redis server v=3.2.5 sha=00000000:0 malloc=jemalloc-4.0.3 bits=64 build=7a2c9bceaa71a5cb
ownCloud version: (see ownCloud admin page)
ownCloud 10.0.9 (stable) at the moment
the version when the issue occured was 10.0.8.4
Updated from an older ownCloud or fresh install:
ownCloud 9
Where did you install ownCloud from:
ubuntu repo http://download.owncloud.org/download/repositories/production/Ubuntu_16.04/
Signing status (ownCloud 9.0 and above):

Login as admin user into your ownCloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results into https://gist.github.com/ and puth the link here.

https://gist.github.com/hmalessa/c2c90d277eac17f0fc3f9268bca6b478

The content of config/config.php:

Log in to the web-UI with an administrator account and click on
'admin' -> 'Generate Config Report' -> 'Download ownCloud config report'
This report includes the config.php settings, the list of activated apps
and other details in a well sanitized form.

or 

If you have access to your command line run e.g.:
sudo -u www-data php occ config:list system
from within your ownCloud installation folder

*ATTENTION:* Do not post your config.php file in public as is. Please use one of the above
methods whenever possible. Both, the generated reports from the web-ui and from occ config:list
consistently remove sensitive data. You still may want to review the report before sending.
If done manually then it is critical for your own privacy to dilligently
remove *all* host names, passwords, usernames, salts and other credentials before posting.
You should assume that attackers find such information and will use them against your systems.

{
"system": {
"updatechecker": false,
"instanceid": "oc7cabrr7t9d",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [
"REMOVED SENSITIVE VALUE",
"REMOVED SENSITIVE VALUE"
],
"datadirectory": "/var/www/owncloud/data",
"overwrite.cli.url": "http://REMOVED SENSITIVE VALUE/owncloud",
"dbtype": "mysql",
"version": "10.0.9.5",
"dbname": "owncloud",
"dbhost": "localhost",
"dbtableprefix": "oc_",
"dbuser": "REMOVED SENSITIVE VALUE",
"dbpassword": "REMOVED SENSITIVE VALUE",
"logtimezone": "UTC",
"installed": true,
"mail_smtpmode": "smtp",
"mail_smtpsecure": "ssl",
"mail_from_address": "REMOVED SENSITIVE VALUE",
"mail_domain": "REMOVED SENSITIVE VALUE",
"mail_smtphost": "REMOVED SENSITIVE VALUE",
"mail_smtpport": "25",
"activity_expire_days": 60,
"memcache.local": "\OC\Memcache\Redis",
"filelocking.enabled": "true",
"memcache.distributed": "\OC\Memcache\Redis",
"memcache.locking": "\OC\Memcache\Redis",
"redis": {
"host": "localhost",
"port": 6379,
"timeout": 0,
"dbindex": 0
},
"theme": "",
"loglevel": 2,
"maintenance": false,
"preview_max_scale_factor": 1,
"enabledPreviewProviders": {
"0": "OC\Preview\PNG",
"1": "OC\Preview\JPEG",
"2": "OC\Preview\GIF",
"11": "OC\Preview\Illustrator",
"12": "OC\Preview\Postscript",
"13": "OC\Preview\Photoshop",
"14": "OC\Preview\TIFF"
}
}
}

List of activated apps:

If you have access to your command line run e.g.:
sudo -u www-data php occ app:list
from within your ownCloud installation folder.

Enabled:

• activity: 2.3.7
• comments: 0.3.0
• configreport: 0.1.1
• dav: 0.3.2
• federatedfilesharing: 0.3.1
• federation: 0.1.0
• files: 1.5.1
• files_external: 0.7.1
• files_pdfviewer: 0.9.0
• files_sharing: 0.10.1
• files_texteditor: 2.2.1
• files_trashbin: 0.9.1
• files_versions: 1.3.0
• files_videoplayer: 0.9.8
• firstrunwizard: 1.1
• gallery: 16.1.0
• market: 0.2.5
• notifications: 0.3.4
• provisioning_api: 0.5.0
• systemtags: 0.3.0
• templateeditor: 0.3.1
• updatenotification: 0.2.1
  Disabled:
• encryption
• external
• gallery.orig
• user_external

Are you using external storage, if yes which one: local/smb/sftp/...
no

Are you using encryption: yes/no
no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

LDAP configuration (delete this part if not used)

With access to your command line run e.g.:
sudo -u www-data php occ ldap:show-config
from within your ownCloud installation folder

Without access to your command line download the data/owncloud.db to your local
computer or access your SQL server remotely and run the select query:
SELECT * FROM `oc_appconfig` WHERE `appid` = 'user_ldap';


Eventually replace sensitive data as the name/IP-address of your LDAP server or groups.

Client configuration

Browser:

Operating system:

Logs

Web server error log

Insert your webserver log here

ownCloud log (data/owncloud.log)

Insert your ownCloud log here

link expiry of first shared link was 2018-08-15.

OK: access to first shared link without password (I assume this was a search engine request)
{"reqId":"Rim7OrExgRzGPfrZUbQh","level":3,"time":"2018-08-14T19:52:16+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"POST","url":"/owncloud/index.php/apps/gallery/s/pUnWVKTJNw6nlSW","message":"Exception: Wrong password (401)"}

OK: same request like above but after expiration (shared link was removed)
{"reqId":"2ejcSurExp8O8piw5x19","level":3,"time":"2018-08-15T09:51:27+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"GET","url":"/owncloud/index.php/apps/gallery/s/pUnWVKTJNw6nlSW","message":"Exception: Unspecified share exception (404)"}

FAIL: DeleteOrphanedSharesJob with dead lock
{"reqId":"cpRcMxZW0eDrUPD5Cbwf","level":3,"time":"2018-08-21T09:00:14+00:00","remoteAddr":"","user":"--","app":"core","method":"--","url":"--","message":"Error while running background job (class: OCA\Files_Sharing\DeleteOrphanedShar
esJob, arguments: ): {"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'DELETE FROM oc_share WHERE item_type in ('file', 'folder') AND NOT EXISTS (SELECT fileid FROM oc_filecache WHERE file_source = fileid)':\n\nSQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction","Code":0,"Trace":"#0 \/var\/www\/owncloud\/lib
\/composer\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/DBALException.php(128): Doctrine\\DBAL\\Driver\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\DBAL\\Driver\\PDOException))\n#1 \
/var\/www\/owncloud\/lib\/composer\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Connection.php(1015): Doctrine\\DBAL\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\DBAL\\Driver\\PDOMySql\\Driver),
Object(Doctrine\\DBAL\\Driver\\PDOException), 'DELETE FROM oc...', Array)\\n#2 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/DB\\\/Connection.php(210): Doctrine\\\\DBAL\\\\Connection->executeUpdate('DELETE FROM oc...', Array
, Array)\n#3 \/var\/www\/owncloud\/apps\/files_sharing\/lib\/DeleteOrphanedSharesJob.php(61): OC\\DB\\Connection->executeUpdate('DELETE FROM `oc...')\n#4 \/var\/www\/owncloud\/lib\/private\/BackgroundJob
\/Job.php(57): OCA\\Files_Sharing\\DeleteOrphanedSharesJob->run(NULL)\n#5 \/var\/www\/owncloud\/lib\/private\/BackgroundJob\/TimedJob.php(53): OC\\BackgroundJob\\Job->execute(Object(OC\\BackgroundJob\\JobLi
st), Object(OC\\Log))\n#6 \/var\/www\/owncloud\/cron.php(121): OC\\BackgroundJob\\TimedJob->execute(Object(OC\\BackgroundJob\\JobList), Object(OC\\Log))\n#7 {main}","File":"\/var\/www\/owncloud\/lib\
/composer\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/AbstractMySQLDriver.php","Line":115}"}

FAIL: first request of second shared link addressing the same folder like first link. Link was created on 2018-07-23 with expiration in 2019
{"reqId":"A1QTyldRk8aHf3Cz4VMj","level":3,"time":"2018-08-21T18:20:27+00:00","remoteAddr":"192.168.122.180","user":"--","app":"gallery","method":"GET","url":"/owncloud/index.php/apps/gallery/s/pSHcQ80GsuRCb4J","message":"Exception: Unspecified share exception (404)"}

The query "select * from oc_share where share_type=3" has no results (tested at 2018-08-21 22:00)

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log 
c) ...

[ownclouders]

GitMate.io thinks the contributors most likely able to help are @ownclouders, and @PVince81.

Possibly related issues are #22327 (Allow multiple link shares), #7813 ([FEATURE REQUEST] Share link with multiple users has different link), #11097 (Shared links expire with no expiry date set), #15022 (Share link hangs), and #31129 (Add missing public link share dialog features).

[PVince81]

From what I see in the code the deletion should only affect shares that are actually expired: https://github.com/owncloud/core/blob/v10.0.9/lib/private/Share20/Manager.php#L1129

There is a loop that checks for each share.

@hmalessa how far apart were the expiration dates ? Is this reproducible consistently ? Are maybe time zones involved where maybe you believed that the second link was still valid but a time zone difference might have deleted it early ? (see #8363 for the timezone issue)

I've set up a test case on my server (10.0.9) and will see whether the observed problem happens there as well.

[PVince81]

I had set a link to expire today and another next week. Today I opened the share panel for the folder in question and I can see that today's link did disappear, but the one from next week is still there.

So no problem observed so far...

[PVince81]

@hmalessa when you say "wait until expiration date of first link" there must be a step after that: are you opening the expired link ? or are you opening the share panel as I did ?

because the action you did after that is important as the expiry logic is triggered only the next time the link is accessed or listed

[leblanc217]

@ownclouders, and @PVince81

Adding to this old thread. We also expirenced this behaviour today. We have directories with multiple public shares, one expiring the day of the issue (old link) and another expiring two weeks into the future (new link). When the old link expired, it also removed the new link we uploaded that day.

New to GitHub so any information required please let me know and I'll provide it.

Showing share activity, note two shares uploaded and only one expired:

Showing no public links after first one expired. Should be at least one link as there are not two expire events:

[phil-davis]

@dpakach we did some tests for link expiry recently. IMO we should now be able to add a test scenario where there are 2 public link shares of a resource. Then we put the expiry of one of them into the past, and access that "past-expired" link share. That should cause the "past-expired" link share to disappear. But the other link share that expires in the future should still exist.

Such a test scenario should be able to demonstrate this problem - and then someone can fix it.

I added this to the QA Test Automation project so that we can demonstrate the issue.

[phil-davis]

We added a test scenario in #39100 and that passes. I tried a few permutations of that locally, and could not find a sequence that failed. @SwikritiT tried locally to make public links with different expiry dates, and let one expire "naturally" overnight. And still we don't see the problem.

The other place that could be a problem is https://github.com/owncloud/core/blob/master/apps/files_sharing/lib/ExpireSharesJob.php - maybe if that runs, and finds a public link share that has expired, that it accidentally removes all the public link shares of the resource. But looking at the code, it explicitly queries the database only for shares with expiry before "today". And it loops through that list deleting the shares.

Before PR #38775 if there was a share that expired yesterday (1), and a share that expires today (2), and a share that expires tomorrow (3), then both (1) and (2) would have been deleted. After that PR (i.e. in 10.8.0 release) only (1) gets deleted.

Please try with 10.8.0 and let us know if you can still reproduce the problem.