Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Request] IdMap/IdMap2 for IndexBinary #780

Closed
bduclaux opened this issue Apr 10, 2019 · 2 comments
Closed

[Feature Request] IdMap/IdMap2 for IndexBinary #780

bduclaux opened this issue Apr 10, 2019 · 2 comments
Labels
enhancement

Comments

@bduclaux
Copy link

Hello,

As far as I know, there is currently no way to use an IndexIDMap / IndexIDMap2 with binary indexes, as the IDMap classes derive from the Index class, and not the IndexBinary class.

Such code does not compile:

faiss::IndexBinaryHNSW * hnsw=new faiss::IndexBinaryHNSW(dim, 32);
faiss::IndexIDMap2 * hnsw_index=new faiss::IndexIDMap2(hnsw);

It would be nice to have IndexBinaryIDMap / IndexBinaryIDMap2 classes in a future release, with serialization support in index_io.cpp as for other indexes.

The current workaround is to store mappings between ids and record numbers in a separate binary file.

Best
@mdouze
Copy link
Contributor

mdouze commented Apr 11, 2019

Yes we are progressively extending the meta-indexes to the binary indexes as well.

@beauby beauby mentioned this issue Jun 13, 2019
Merged
beauby pushed a commit that referenced this issue Jun 19, 2019
Facebook sync (Jun 2019) (#862)
3896b12 
Bugfixes:
- slow scanning of inverted lists (#836).

Features:
- add basic support for 6 new metrics in CPU `IndexFlat` and `IndexHNSW` (#848);
- add support for `IndexIDMap`/`IndexIDMap2` with binary indexes (#780).

Misc:
- throw python exception for OOM (#758);
- make `DistanceComputer` available for all random access indexes;
- gradually moving from `long` to `int64_t` for portability.
@mdouze
Copy link
Contributor

mdouze commented Jun 25, 2019

Implemented in 1.5.3

@mdouze mdouze closed this as completed Jun 25, 2019
@luochenxi luochenxi mentioned this issue Jul 4, 2019
Closed
6 tasks
@mqudsi mqudsi mentioned this issue Feb 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@beauby @mdouze @bduclaux