Ignore duplicate inserts in file cache and mime type #5442
Conversation
| catch (\Doctrine\DBAL\DBALException $e){ | ||
| \OC_Log::write('core', 'Exception during file cache insertion: ' . $e->getmessage(), \OC_Log::DEBUG); | ||
| // in case of duplicate insert error, retrieve the existing file data | ||
| $fileId = self::getId($file); |
There was a problem hiding this comment.
For some reason getId() returns -1, as the select inside doesn't return any results for some mysterious reason.
|
Test passed. |
| catch (\Doctrine\DBAL\DBALException $e){ | ||
| \OC_Log::write('core', 'Exception during mimetype insertion: ' . $e->getmessage(), \OC_Log::DEBUG); | ||
| // in case of duplicate insert error, retrieve the existing mime type id | ||
| $result = \OC_DB::executeAudited('SELECT `id` FROM `*PREFIX*mimetypes` WHERE mimetype=?', array($mime)); |
There was a problem hiding this comment.
should be mimetype - maybe this is why the query is not returning a proper result set?
There was a problem hiding this comment.
There was a problem hiding this comment.
I mean the backticks are missing on the where
There was a problem hiding this comment.
I had the issue with "no results" in the other block, the one wrapping the insert into oc_filecache, not the mimetype one.
When two scanning processed run at the same time, for example when scan.php and list.php were called at the same time on an external storage mountpoint, duplicate insertion errors can occurs. These errors are now logged and ignored. Since both scans are running in parallel transactions, they don't see each other's changes directly in the DB which can cause duplicate insertion errors for mime types as well, but those mime types can't be selected yet. The solution to this is to force-reload the mimetypes list after the transaction is finished. Fixes #5199
|
I've updated the PR description that explains the current approach. Please review this approach: @karlitschek @icewind1991 @DeepDiver1975 @bantu |
|
Test passed. |
|
Test passed. |
|
👍 |
|
@jancborchardt glad to read that! |
|
Test passed. |
|
👍 |
Ignore duplicate inserts in file cache and mime type
Catch duplicate insertion errors while scanning files
When two scanning processed run at the same time, for example when
scan.php and list.php were called at the same time on an external
storage mountpoint, duplicate insertion errors can occurs.
These errors are now logged and ignored.
Since both scans are running in parallel transactions, they don't see
each other's changes directly in the DB which can cause duplicate
insertion errors for mime types as well, but those mime types can't be
selected yet. The solution to this is to force-reload the mimetypes list
after the transaction is finished.
Fixes #5199