[BUG]: Missing extra meta data in the API document/raw-text

[byzheng]

How are you running AnythingLLM?

AnythingLLM desktop app

What happened?

I am following Anything LLM developer API documentation and using document/raw-text.

This is the data I post

{
  "textContent": "This is the raw text that will be saved as a document in AnythingLLM.",
  "metadata": {
    "title": "title",
    "key": "value"
  }
}

A new document with id 99502983-20fa-4cd1-bef5-e17cd0c56dae is added.

Then I used /v1/document/{docName} to retrieve document with name raw-title-99502983-20fa-4cd1-bef5-e17cd0c56dae.json, but cannot find the key and value

Are there known steps to reproduce?

No response

[timothycarambat]

This endpoint value is really for upserting simple text, the custom metadata that does not comply to our uniform schema will be omitted - this entire endpoint was really just a simple extension of the endpoint since the API process to upload a document then embed was very clunky and this streamlined that process.

anything-llm/collector/processRawText/index.js

Lines 13 to 35 in 7b42a67

	const METADATA_KEYS = {
	possible: {
	url: ({ url, title }) => {
	let validUrl;
	try {
	const u = new URL(url);
	validUrl = ["https:", "http:"].includes(u.protocol);
	} catch { }
	if (validUrl) return `web://${url.toLowerCase()}.website`;
	return `file://${stripAndSlug(title)}.txt`;
	},
	title: ({ title }) => `${stripAndSlug(title)}.txt`,
	docAuthor: ({ docAuthor }) => { return typeof docAuthor === 'string' ? docAuthor : 'no author specified' },
	description: ({ description }) => { return typeof description === 'string' ? description : 'no description found' },
	docSource: ({ docSource }) => { return typeof docSource === 'string' ? docSource : 'no source set' },
	chunkSource: ({ chunkSource, title }) => { return typeof chunkSource === 'string' ? chunkSource : `${stripAndSlug(title)}.txt` },
	published: ({ published }) => {
	if (isNaN(Number(published))) return new Date().toLocaleString();
	return new Date(Number(published)).toLocaleString()
	},
	}
	}

So, this behavior is intentional.