Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the used module numpydb to diskcache for stamp storage #42

Open
BrunoSanchez opened this issue Aug 27, 2020 · 5 comments
Open

Update the used module numpydb to diskcache for stamp storage #42

BrunoSanchez opened this issue Aug 27, 2020 · 5 comments

Comments

@BrunoSanchez
Copy link
Collaborator

BrunoSanchez commented Aug 27, 2020

There is an implementation for a numpy array storage, that uses pickle.
This is critical so we don't keep in memory stamps of every star.
The current implementation though, requires the creation of files, which for many reasons can persist later as garbage in the local directory (see #11).

Using diskcache library could avoid such issue, and clean the base code of additional modules.

@BrunoSanchez
Copy link
Collaborator Author

A comment on numpyDB: the code is public in two places

Maybe it makes sense to make scitools a dependency, it is in PyPI and we could avoid mantaining numpyDB.

@martinberoiz
Copy link
Member

Yes, judging by the last commit (esheldon/numpydb@5ccce22) and lack of versions in esheldon's repo, I would go with scitools copy.

@leliel12
Copy link
Collaborator

leliel12 commented Oct 13, 2020 via email

@leliel12
Copy link
Collaborator

leliel12 commented Oct 13, 2020 via email

@BrunoSanchez
Copy link
Collaborator Author

Scitools is a big library and last commit is from 2016. I don't know if it supports Python > 2.6

esheldon's repo is only about NupyDB, latest commit is from 2019 and there is some extra functionality which I don't use.
However he mentions some seg fault in his latest commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants