Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sources indexing in MongoDB #73

Open
6 tasks
caviri opened this issue Apr 8, 2024 · 0 comments
Open
6 tasks

Sources indexing in MongoDB #73

caviri opened this issue Apr 8, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@caviri
Copy link
Contributor

caviri commented Apr 8, 2024

Context
In order to monitor specific data sources we need to implement a way to monitor the status of these sources. The user can define a set of data sources (data-endpoints?) that are going to be checked automatically for updates.

This can be used to trigger a new execution when the source is updated.

Proposed solution
Separate indexing from monitoring. Sources are a new mongoDB collection that keeps the link to the data source, the type, the last time it was accessed and other metadata of relevance. This collection can be at the same level of components.

Acceptance criteria

  • New MongoDB collection called sources
  • sources schema.
  • Add a new source in CLI
  • Add a new source in GUI
  • Listing sources in CLI
  • Listing sources in GUI
@caviri caviri added the enhancement New feature or request label Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant