PDF Files that were changed by app external to Obsidian do not trigger a new text extraction

[rwaldspur]

Hi,

In my workflow, I'm updating notes in note-taking apps (e.g. Notability and GoodNotes). Each time the documents in these apps are changed, I have them trigger a new backup of that note as a PDF into an Obsidian Vault. Notably, this PDF comes with updated OCR text.

The new backup replaces the old version of the file in the vault -- same file name, but different mod date.

Unfortunately, since the change did not originate in Obsidian, it seems that the triggering of a new extraction does not happen, as presumably the internal event bus does not generate an event for OmniSearch to see.

I can use an external file watcher in macOS to trigger events when the PDFs are changed, but I can't actually figure out a simple way to either: 1) Have Obsidian trigger a modification event for those files (e.g. using URI command or other), or 2) Similarly, have OmniSearch / TextExtractor trigger a new text extraction, replacing the old.

If one right-clicks on a PDF file that has been updated, and chooses "Text Extractor... clear cache for this file" one can remove the cache manually, and then re-start Obsidian. This then triggers a rebuild.

However, I'd need to know which files changed, and then manually take care of this for each and every file.

Any ideas or how to overcome this problem?

OmniSearch is doing a great job, it is almost seamless for me.

[scambier]

Hello,

In theory, what you're describing should already work, but I don't really work with PDFs, and actually never tried to modify one for Obsidian. If you update a PDF and then just restart Obsidian (without clearing the cache), does it work?

[rwaldspur]

Thank you.

No, without clearing the cache, even with a restart of Obsidian, it won't update the text extraction, and thereby the search repository.

I just did a new test, where I took a PDF file, and had OmniSearch index it. I was able to find material in the PDF without issue. I then replaced that PDF in the macOS Finder with an entirely different PDF, but named the same. This did not affect the extracted text. It remained the same, even after restart of Obsidian.

Does TextExtractor / OmniSearch depend on the EventBus to flag a modified file? If so, I'm wondering if an event is not being generated because the (PDF) file is being modified externally.

BTW, in the above example -- when I replace a PDF with a different one, but named the same -- but this time instead rename the file in the macOS Finder to yet a new name, it triggers a new text extraction for that new file name, and I can successsfully use OmniSearch with this 'replaced' PDF. However, if I then afterwards change the name of this 'replaced' PDF back to the original name, the cached data refers instead to the original text extraction, which is no longer the correct data.

Again, thank you for any insights here.

[scambier]

No, without clearing the cache, even with a restart of Obsidian, it won't update the text extraction, and thereby the search repository.

It's a bug then, I'll take a look at it asap 👍

Does TextExtractor / OmniSearch depend on the EventBus to flag a modified file? If so, I'm wondering if an event is not being generated because the (PDF) file is being modified externally.

It relies on Obsidian's watcher

obsidian-omnisearch/src/main.ts

Lines 92 to 100 in 5066b01

	this.registerEvent(
	this.app.vault.on('modify', async file => {
	if (isFileIndexable(file.path)) {
	logDebug('Updating file', file.path)
	await cacheManager.addToLiveCache(file.path)
	NotesIndex.markNoteForReindex(file)
	}
	})
	)

Updated files are flagged, and reindexed the next time Omnisearch is opened. I should review this code as it's a bit older, but I don't think the issue lies there: if the PDF cache is not updated after a restart, the problem isn't (probably) with the watcher, but within Text Extractor.

[rwaldspur]

Thanks so very much for taking a look at this.