WebDAV yields empty results when encountering restricted content in SMB external storage (patch included) #24893

didierm · 2020-12-29T20:40:43Z

Steps to reproduce

server : create SMB/CIFS external storage share (settings : log-in credentials saved in database, check for changes once every direct access)
client : (tested with Windows10 Explorer) map NC share to Windows drive letter via WebDAV URL
client : open drive letter
if the NC SMB folder scan encounters a folder with restricted access rights (i.e. forbidden access due to restrictive ACLs), the NC server throws a fatal ForbiddenException error, and aborts the directory scan, which results in an "Empty Folder" at the client.

Expected behaviour

The folder scan should skip the restricted folder(s) and provide the client with the readable contents of the share.

Actual behaviour

The folder scan aborts, and returns an empty folder list.
This happens at both the top share level, or later at any subfolder level, if restricted content is encountered.

Server configuration

Operating system:
CentOS Linux release 7.9.2009 (Core)

Web server:
Apache 2.4.6

Database:
PostgreSQL 9.2.24

PHP version:
PHP 7.3.25

Nextcloud version: (see Nextcloud admin page)
versions 19.0.4 and 20.0.4

Are you using external storage, if yes which one: local/smb/sftp/...
SMB/CIFS

Are you using encryption:
no

Are you using an external user-backend, if yes which one:
LDAP

Client configuration

Browser:
Windows Explorer

Operating system:
Windows10

Logs

Nextcloud log (data/nextcloud.log)

Nextcloud log

[webdav] Fatal: Icewind\SMB\Exception\ForbiddenException: Invalid request for /personeel (ForbiddenException) at <<closure>>

 0. /opt/nextcloud.staging/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php line 66
    Icewind\SMB\Exception\Exception::fromMap({1: "Icewind\\SM ... "}, 13, "/personeel")
 1. /opt/nextcloud.staging/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php line 78
    Icewind\SMB\Native\NativeState->handleError("/personeel")
 2. /opt/nextcloud.staging/apps/files_external/3rdparty/icewind/smb/src/Native/NativeState.php line 109
    Icewind\SMB\Native\NativeState->testResult(false, "smb://obfusc/T/personeel")
 3. /opt/nextcloud.staging/apps/files_external/3rdparty/icewind/smb/src/Native/NativeShare.php line 92
    Icewind\SMB\Native\NativeState->opendir("smb://obfusc/T/personeel")
 4. /opt/nextcloud.staging/apps/files_external/lib/Lib/Storage/SMB.php line 238
    Icewind\SMB\Native\NativeShare->dir("personeel")
 5. /opt/nextcloud.staging/apps/files_external/lib/Lib/Storage/SMB.php line 610
    OCA\Files_External\Lib\Storage\SMB->getFolderContents("personeel")
 6. <<closure>>
    OCA\Files_External\Lib\Storage\SMB->getDirectoryContent("personeel")
 7. /opt/nextcloud.staging/lib/private/Files/Cache/Scanner.php line 408
    iterator_to_array(Generator {})
 8. /opt/nextcloud.staging/lib/private/Files/Cache/Scanner.php line 388
    OC\Files\Cache\Scanner->handleChildren("personeel", false, 3, 29468776, true, 0)
 9. /opt/nextcloud.staging/lib/private/Files/Cache/Scanner.php line 340
    OC\Files\Cache\Scanner->scanChildren("personeel", false, 3, 29468776, true)
10. /opt/nextcloud.staging/lib/private/Files/View.php line 1345
    OC\Files\Cache\Scanner->scan("personeel", false)
11. /opt/nextcloud.staging/lib/private/Files/View.php line 1389
    OC\Files\View->getCacheEntry(OCA\Files_Antivi ... l}, "personeel", "/IRC FileServer ... l")
12. /opt/nextcloud.staging/apps/dav/lib/Connector/Sabre/Directory.php line 330
    OC\Files\View->getFileInfo("/Didier.Moens/f ... l", false)
13. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 799
    OCA\DAV\Connector\Sabre\Directory->getQuotaInfo()
14. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/PropFind.php line 96
    Sabre\DAV\CorePlugin->Sabre\DAV\{closure}("*** sensitive parameters replaced ***")
15. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 802
    Sabre\DAV\PropFind->handle("{DAV:}quota-used-bytes", Closure {})
16. /opt/nextcloud.staging/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->propFind(Sabre\DAV\PropFind {}, OCA\DAV\Connector\Sabre\Directory {})
17. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 1063
    Sabre\DAV\Server->emit("propFind", [Sabre\DAV\PropF ... }])
18. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 989
    Sabre\DAV\Server->getPropertiesByNode(Sabre\DAV\PropFind {}, OCA\DAV\Connector\Sabre\Directory {})
19. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 1678
    Sabre\DAV\Server->getPropertiesIteratorForPath("files/Didier.Mo ... )", [], 1)
20. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 1661
    Sabre\DAV\Server->writeMultiStatus(Sabre\Xml\Writer ... ]}, Generator {}, false)
21. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/CorePlugin.php line 363
    Sabre\DAV\Server->generateMultiStatus(Generator {}, false)
22. /opt/nextcloud.staging/3rdparty/sabre/event/lib/WildcardEmitterTrait.php line 89
    Sabre\DAV\CorePlugin->httpPropFind(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
23. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 474
    Sabre\DAV\Server->emit("method:PROPFIND", [Sabre\HTTP\Requ ... }])
24. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 251
    Sabre\DAV\Server->invokeMethod(Sabre\HTTP\Request {}, Sabre\HTTP\Response {})
25. /opt/nextcloud.staging/3rdparty/sabre/dav/lib/DAV/Server.php line 319
    Sabre\DAV\Server->start()
26. /opt/nextcloud.staging/apps/dav/lib/Server.php line 332
    Sabre\DAV\Server->exec()
27. /opt/nextcloud.staging/apps/dav/appinfo/v2/remote.php line 35
    OCA\DAV\Server->exec()
28. /opt/nextcloud.staging/remote.php line 167
    require_once("/opt/nextcloud. ... p")

More information

this issue seems to have been introduced in the course of NC19 (if memory serves well, this did not occur with NC18).
this issue is also reported here by another user : https://help.nextcloud.com/t/webdav-smb-drive-subfolder-permission-error/101511, and is mitigated by enabling the external share setting [Verify ACL access when listing files], which unfortunately induces a severe performance impact on an already less than stellar WebDAV.

Fix

By tracing the calls in the above quoted Exception log, it was deduced the directory scan exits the function at :

server/apps/files_external/lib/Lib/Storage/SMB.php

Line 235 in 4c01f0d

protected function getFolderContents($path): iterable {

Taking inspiration from the merged patch in the very analogous issue #7332, the following change fixes the issue :

server/apps/files_external/lib/Lib/Storage/SMB.php

Line 238 in 4c01f0d

$files = $this->share->dir($path);

try {
	$files = $this->share->dir($path);
} catch (ForbiddenException $e) {
	return false;
}

I am not sure whether it's appropriate, but you may probably want to catch (NotFoundException $e) too.

The text was updated successfully, but these errors were encountered:

kesselb · 2020-12-30T18:52:39Z

the following change fixes the issue

does it? The return is type hinted to iterable. false is not an iterable.

didierm · 2020-12-30T20:44:41Z

My PHP skills date from > 20 years ago (before all this fancy OO programming, and now get off my lane ;) ).

I am not able to provide any feedback about iterables if my life would depend on it, but the ForbiddenException breakage does not occur any more when encountering restricted content, and the SMB share is now fully accessible with WebDAV, where previously it was not.

Example of an SMB share :

a/
b/
c/

with "b/" being restricted (ACL's applied).

Without the patch :

top level : This folder is empty

With the patch :

top level :

a
b
c

When entering "b/" : This folder is empty

-> I.e. working as expected.

didierm · 2020-12-30T20:46:38Z

Note : next to our production deployment (two 19.0.4 instances, soon to be upgraded to 20.0.4), I have a 20.0.4 sandbox environment, available for testing patches.

lainwir3d · 2021-01-05T11:44:05Z

Same issue with:

CentOS 8.3
Nextcloud 20.0.4
PHP 7.4.13
php-smbclient-1.0.1

Fix from @didierm fixes the issue from me, SMB shares are now accessible again.

Cheers!

hermanntoast · 2021-01-10T13:51:57Z

I have the same issue with my Nextcloud 20.0.4 on Ubuntu 20.04 and PHP 7.4. The fix from @didierm worked for me too!
Thanks!

didierm · 2021-01-27T21:29:20Z

@PVince81 Is the fix in #25313 analogous to the above described issue & attempted patch ?

PVince81 · 2021-01-28T07:37:52Z

@didierm not really. The PR #25313 is to mitigate against a rare situation where oc_filecache has an inconsistency.

@icewind1991 any comments on the patch above ?

juliusknorr · 2021-01-28T16:30:25Z

The issue actually seems to be showing since the merge of #24102 as there the available space was requested additionally in the files propfind request, however the underlying issue with the no permission exception not being cached anywhere when trying to iterate over the directory is something that must have been there before.

I tend to agree that catching the exception inside of the getFolderContents method would be a somehow sane attempt to solve this, however I cannot fully see if that has any other side-effects.

To reproduce the issue basically create a smb share with ACLs applied to only one subdirectory and do not check the "Verify ACL access when listing files" setting.

# smb.conf
[public]                                                                        
   path = /smbmount                                                             
   inherit acls = yes                                        
   browsable = no                                                               
   read only = no                                                               
   guest ok = no                                                  
   delete veto files = yes                                                      
   write list = admin    

mkdir /smbmount/{a,b,c}
setfacl -R -m other:--- /smbmount/b/

juliusknorr · 2021-01-29T11:14:24Z

Testing of #25383 is very welcome, a patch to apply on top of 20.0.5 is also available here:
https://gist.githubusercontent.com/juliushaertl/d7c48af700e3ea44b097719014f27505/raw/1958cf694021fc4dabfaf9565b1ffb284d864cfe/0001-Properly-handle-SMB-ACL-blocking-scanning-a-director.patch

You can apply the patch with the following command inside of your Nextcloud directory:

wget https://gist.githubusercontent.com/juliushaertl/d7c48af700e3ea44b097719014f27505/raw/1958cf694021fc4dabfaf9565b1ffb284d864cfe/0001-Properly-handle-SMB-ACL-blocking-scanning-a-director.patch -o 25383.patch
patch -p1 < 25383.patch

didierm · 2021-01-30T15:49:25Z

@juliushaertl @rullzer
Patch applied to v20.0.6 & issue confirmed fixed.
Thank you !

-> milestone 20.0.7 candidate ?

didierm added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Dec 29, 2020

juliusknorr mentioned this issue Jan 29, 2021

Properly handle SMB ACL blocking scanning a directory #25383

Merged

rullzer closed this as completed in #25383 Jan 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WebDAV yields empty results when encountering restricted content in SMB external storage (patch included) #24893

WebDAV yields empty results when encountering restricted content in SMB external storage (patch included) #24893

didierm commented Dec 29, 2020 •

edited

Loading

kesselb commented Dec 30, 2020

didierm commented Dec 30, 2020

didierm commented Dec 30, 2020

lainwir3d commented Jan 5, 2021

hermanntoast commented Jan 10, 2021

didierm commented Jan 27, 2021 •

edited

Loading

PVince81 commented Jan 28, 2021

juliusknorr commented Jan 28, 2021

juliusknorr commented Jan 29, 2021

didierm commented Jan 30, 2021

WebDAV yields empty results when encountering restricted content in SMB external storage (patch included) #24893

WebDAV yields empty results when encountering restricted content in SMB external storage (patch included) #24893

Comments

didierm commented Dec 29, 2020 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Nextcloud log (data/nextcloud.log)

More information

Fix

kesselb commented Dec 30, 2020

didierm commented Dec 30, 2020

Without the patch :

With the patch :

didierm commented Dec 30, 2020

lainwir3d commented Jan 5, 2021

hermanntoast commented Jan 10, 2021

didierm commented Jan 27, 2021 • edited Loading

PVince81 commented Jan 28, 2021

juliusknorr commented Jan 28, 2021

juliusknorr commented Jan 29, 2021

didierm commented Jan 30, 2021

didierm commented Dec 29, 2020 •

edited

Loading

didierm commented Jan 27, 2021 •

edited

Loading