Filter functionality for Faiss vector store delete operation

[golkir]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Currently, the Faiss document delete method only supports deleting by ids. For example,
await vectorStore.delete({ids: [1,2,3]});
However, it would be nice to be able to delete documents without specifying ids directly, but using some filter condition instead. Such filtering can be done on the document's metadata.

Motivation

When splitting documents into chunks for vector storage, one can keep information about where these chunks came from by setting custom fields in the metadata object. The chunked documents created will have different ids but will share the same metadata value. Just by knowing this value we can filter documents for deletion without knowing their ids.

Proposal (If applicable)

To create a filterThenDelete utility method. This function will filter documents based on metadata keys and call delete method with the array of retrieved ids. It can look something like this:

export type MetadataFilter = {
  [key: string]: any;
}
 /**
   * Method to delete documents.
   * @param filter A filter on documents metadata key/value pairs
   * @returns A promise that resolves when the deletion is complete.
   */

  async filterThenDelete(filter: MetadataFilter) {
    let documentIds = Array.from(this.docstore._docs.entries())
      .filter(([id, doc]: [string, Document]) => {
        return Object.entries(filter).every(([key, value]) => doc.metadata && doc.metadata[key] === value);
      })
      .map(([id, _]: [string, any]) => id);
    if (documentIds == null) {
      throw new Error("No documents match the filter.");
    }
    return await this.delete({ ids: documentIds });
  }