How to read JSON file on llamaindex ts.

[ozgur142]

Hello,

i am working with json files and i would like to embedded them to speak with json files but as i see, json files are not supported on llamaindes TS, is there any way to use json files as input?

Thank you for your answers

[KindOfAScam]

@ozgur142 I've been working with a lot of JSON recently as well, so I added a JSONReader a couple of days ago.
Give it a shot and lmk if something is not working as expected!

[ozgur142]

@KindOfAScam KindOfAScam, thank you for your answer, it is working good for small data but i have some json files with 200k to 2M lines and it is taking too much time (i never waited till the end) soi was wondering if there is a faster way.

I am using nomic-embed-text model from ollama with 768 embedding dim

[KindOfAScam]

@ozgur142 It's a general issue with JSON.parse and large files. The solution would be to integrate some form of JSON streaming (e.g. https://github.com/uhop/stream-json). I have it as a TO-DO but not sure when I get around it, so feel free to take a look at it yourself.

[KindOfAScam]

#1119