-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEAT]: Auto-add and Watch Folder for Document Uploads #1873
Comments
I also feel there is value in having API access to the endpoint where JSON payload can be used to add to the vector DB, |
Will it also handle document changes?. (ea after git pull or after a corection in a file ?. |
This feature is standalone already exists in beta, but you must manually select which file to sync. We do not auto-watch every document due to the probably case of someone wanting to auto-sync 10K files at the same time and that will for sure cause issues |
Can we not at least select a folder to sync? Selecting a file does absolutely nothing for automating the process and would be quite unfortunate. Other RAG systems do offer such options and I do not know if those used to having their RAG auto updating would bother with a system that requires every single file to be added manually. Many applications would require such an option such as knowledge management systems, CRM systems, even generic daily note-taking etc. I do strongly believe that the auto update function would be a critical function to quite a few if not many. Just my two cents. |
As for windows file changes can trigger events, add to it a queue and for standalone it wouldn't matter even if people updated thousands docs which I rhough doubt people would do, but there are many people so perhaps.
When will beta become prod?.
I'm not on discord
Outlook voor Android downloaden<https://aka.ms/ghei36>
…________________________________
From: Asentient ***@***.***>
Sent: Sunday, July 21, 2024 3:15:09 PM
To: Mintplex-Labs/anything-llm ***@***.***>
Cc: PGTBoos ***@***.***>; Manual ***@***.***>
Subject: Re: [Mintplex-Labs/anything-llm] [FEAT]: Auto-add and Watch Folder for Document Uploads (Issue #1873)
Will it also handle document changes?. (ea after git pull or after a corection in a file ?.
This feature is standalone already exists in beta<https://docs.useanything.com/beta-preview/active-features/live-document-sync>, but you must manually select which file to sync. We do not auto-watch every document due to the probably case of someone wanting to auto-sync 10K files at the same time and that will for sure cause issues
Can we not at least select a folder to sync? Selecting a file does absolutely nothing for automating the process and would be quite unfortunate.
Other RAG systems do offer such options and I do not know if those used to having their RAG auto updating would bother with a system that requires every single file to be added manually.
Many applications would require such an option such as knowledge management systems, CRM systems, even generic daily note-taking etc. I do strongly believe that the auto update function would be a critical function to quite a few if not many. Just my two cents.
—
Reply to this email directly, view it on GitHub<#1873 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFQQOG34GXBSIW2IJV7R67LZNOX53AVCNFSM6AAAAABK5JKEOWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBRGYYDOMRZGI>.
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
What would you like to see?
Feature Request: Auto-add and Watch Folder for Document Uploads
Description:
Currently, users need to upload documents individually for the built-in RAG feature. It would be highly beneficial to have an option that allows the system to auto-add all documents from a specified folder and continuously watch the folder for new documents to be added automatically.
Benefits:
Proposed Solution:
Use Case:
This feature would be particularly useful for businesses and researchers who frequently update their document repositories and need these documents to be readily available in AnythingLLM without manual intervention.
The text was updated successfully, but these errors were encountered: