Tools like ChatGPT have a limit on how many tokens can fit into the context. So, let's say you want to converse with it about a specific book – and you want to minimize how much it hallucinates, or 'makes up' stuff about the book. You can't fit the entire book into the chat context. But, if you embed, say, each paragraph of the book as a vector then you can do the following:
+ Take the user message and embed it
+ Do cosine similarity between that message and all of the vectorized passages of the book to pull up the most relevant passages to the user message/question
+ Put just those passages into the context
+ Get a response from ChatGPT using that specific context.
If you happened to see the "Generative Agents" paper that made a splash this week, this is the technique they used as part of the long-term memory retrieval and 'reflection' loop for their 'sims'.
+ Take the user message and embed it
+ Do cosine similarity between that message and all of the vectorized passages of the book to pull up the most relevant passages to the user message/question
+ Put just those passages into the context
+ Get a response from ChatGPT using that specific context.
If you happened to see the "Generative Agents" paper that made a splash this week, this is the technique they used as part of the long-term memory retrieval and 'reflection' loop for their 'sims'.