reference: https://github.com/arbabenko/GNOIMI
The implementation of the (Generalized) Non-Orthogonal Inverted Multi-Index structure from the paper "Efficient Indexing of Billion-Scale Datasets of Deep Descriptors", CVPR 2016.
The code requires Yael library (deps intel mkl) and Intel MKL library. training_gnoimi.cpp contains the code of the codebooks training. searching_gnoimi.cpp contains the code of the index searching.
now, this lab is only compiled on wsl-ubuntu 22.04.
- install intel-mkl
- compile yael based on intel-mkl.
- place headers and libs to proper position.
git clone git@github.com:BismarckDD/gnoimi-impl.git
cd gnoimi-impl
mkdir build && cd build && cmake ..
make -j12
- fix downloadDeep1B.py to python3 style.
- Use Intel MKL instead of cblas, lapack and OpenBlas to rewrite the code.
- Extract common headers to common.h.
- Rename the cpp files, variable and function name according to Google's cpp format.
- Use marco to rewrite thread work assign and join, thus the code seems to be concise. WARNING: the correctness is still to be tested further. Don't use it in production environment unless you know what you are doing.
/*
cblas_saxpy(N, alpha, X, incX, Y, incY)
=> Y = sum(alpha * X[i] + Y[i])
cblas_sdot(N, X, incX, Y, incY)
=> Y =sum(X[i] * Y[i])
*/
sift feature: dim = 128 gist feature: dim = 960