How return id from ConvexVectorStore.fromDocuments

[teomikitenko]

import { v } from "convex/values";
import { action } from "../_generated/server";
import { Document } from "@langchain/core/documents";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { CacheBackedEmbeddings } from "langchain/embeddings/cache_backed";
import { ConvexKVStore } from "@langchain/community/storage/convex";
import { ConvexVectorStore } from "@langchain/community/vectorstores/convex";
import { OpenAIEmbeddings } from "@langchain/openai";

export const loadEmbeddings = action({
  args: {
    text: v.string(),
  },
  handler: async (ctx, { text }) => {
    const textSplitter = new RecursiveCharacterTextSplitter({
      chunkSize: 1000,
      chunkOverlap: 200,
    });
    const splitDocs = await textSplitter.splitDocuments([
      new Document({ pageContent: text }),
    ]);

    const embeddings = new CacheBackedEmbeddings({
      underlyingEmbeddings: new OpenAIEmbeddings(),
      documentEmbeddingStore: new ConvexKVStore({ ctx }),
    });
    await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx });
  },
});

I write embedding using const vectoreStore = await ConvexVectorStore.fromDocuments(splitDocs, embeddings, { ctx });` how do I determine the ID provided by the database for this embed . My application use this ID for search current embeddings - but i dont see information about this in documentation

using LangchainJS in Next.js and Convex
nodejs -v20.15.1
windows 10

[dosubot[bot]]

To determine the ID provided by the database for the embedding created using ConvexVectorStore.fromDocuments, you can use the addDocuments method and provide your own IDs. If you do not provide IDs, the system will generate UUIDs for you. Here is an example of how you can retrieve the IDs:

await withVectorStore(async (vectorStore: MomentoVectorIndex) => {
  const pageContent = faker.lorem.sentence(5);

  // Add documents without providing IDs, so UUIDs will be generated
  await vectorStore.addDocuments([{ pageContent, metadata: {} }]);
  await sleep();
  
  // Perform a similarity search to retrieve the document
  const results = await vectorStore.similaritySearch(pageContent, 1);
  
  // The results will contain the document with the generated ID
  console.log(results);
});

If you want to provide your own IDs, you can do so like this:

await withVectorStore(async (vectorStore: MomentoVectorIndex) => {
  const pageContent = faker.lorem.sentence(5);
  const documentId = "custom-id";

  // Add documents with a custom ID
  await vectorStore.addDocuments([{ pageContent, metadata: {} }], {
    ids: [documentId],
  });
  await sleep();
  
  // Perform a similarity search to retrieve the document
  const results = await vectorStore.similaritySearch(pageContent, 1);
  
  // The results will contain the document with the provided ID
  console.log(results);
});

In both cases, the results will contain the document along with its ID, which you can then use to determine the ID provided by the database.

Additionally, if you are using the MongoDBAtlasVectorSearch from the langchainjs library, you can use the addDocuments method which internally calls addVectors to add documents and retrieve their IDs. Here is an example:

const embeddings = ...; // Your embeddings instance
const docs = [{ pageContent: "example content", metadata: {} }];
const dbConfig = { collection: yourMongoDBCollection, apiKey: "your-api-key" };

const vectorStore = await MongoDBAtlasVectorSearch.fromDocuments(docs, embeddings, dbConfig);
const ids = await vectorStore.addDocuments(docs);

console.log(ids); // This will print the IDs of the added documents

In this example, the addDocuments method will return the IDs of the added documents, which you can then use to determine the ID provided by the database ^[1][2][3].

To continue talking to Dosu, mention @dosu.