This is how RAG pipelines work. You chunk up text and then embed it, which gives you a vector representation of the text. You store that in a database keys in the vector Then when you want to inject relevant information for a prompt, you embed the prompt to get a vector, and then dump the a handful of most similar chunks into context to provide additional information.
So this semantic similarity question has been in my mind for a while to answer the question of what names would be the most “intuitive” for an LLM.
This is how RAG pipelines work. You chunk up text and then embed it, which gives you a vector representation of the text. You store that in a database keys in the vector Then when you want to inject relevant information for a prompt, you embed the prompt to get a vector, and then dump the a handful of most similar chunks into context to provide additional information.
So this semantic similarity question has been in my mind for a while to answer the question of what names would be the most “intuitive” for an LLM.
Does it work? shrug I guess we’ll find out.