Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize query for unscanned files #364

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

zero0cool0
Copy link

If my interpretation of the code in BackgroundScanner is correct, getNodeForFile() relies on the existence of an existing mount to determine the Node for a given fileId.

Sometimes, this fails. Specifically, this line is returning an empty array.
$cachedMounts = $this->userMountCache->getMountsForFileId($fileId)

Tracking this further, I found that no corresponding mount can be found for some fileId that are present in oc_filecache when joining oc_mounts through oc_storage.

Why is this even a problem, other than creating many log entries?

If the number of such entries in oc_filecache is large then the entire batch contains non-scannable entries, starving the scanning of other entries in oc_filecache that have a corresponding mount.

My PR proposes to only process files whose fileId that can be related to an existing mount.

More details:
I have a lot of entries in oc_filecache that look like this:

MariaDB [nextcloud]> select fileid,storage,path,size from oc_filecache where fileid=1044856;
+---------+---------+---------------------------------------------------------------+------+
| fileid  | storage | path                                                          | size |
+---------+---------+---------------------------------------------------------------+------+
| 1044856 |      30 | appdata_ocp617nvaw61/preview/2/7/b/3/b/c/9/667010/256-154.png | 7476 |
+---------+---------+---------------------------------------------------------------+------+

These entries correctly relate to an entry in oc_storages:

+------------+----------------------------+-----------+--------------+
| numeric_id | id                         | available | last_checked |
+------------+----------------------------+-----------+--------------+
|         30 | local::/var/www/html/data/ |         1 |         NULL |
+------------+----------------------------+-----------+--------------+

However, no mount can be found:

MariaDB [nextcloud]> select * from oc_mounts where storage_id=30;
Empty set (0.000 sec)

Signed-off-by: Patrick Fischer <mail@patrickfischer.ch>
Signed-off-by: Patrick Fischer <mail@patrickfischer.ch>
@zero0cool0 zero0cool0 closed this Jul 29, 2024
Signed-off-by: Patrick <zero0cool0@users.noreply.github.com>
Copy link

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants