Why Retrieval Is Harder Than Generation (And What to Do About It)
9
everyone focuses on model quality but retrieval is where most ai systems fail. heres the research
for context – arxiv summaries are the only way i stay current anymore
drop your experiences below
6 replies
6 Replies
Join the discussion.
Log In to Reply
25
chunking strategy matters way more than people realize. fixed-size chunks kill retrieval quality. semantic chunking by paragraph or idea boundary makes a huge difference in what actually gets pulled back.
1
hmm i see what you mean but reranking after initial retrieval is something a lot of teams skip entirely. running a cross-encoder on the top 20 results before passing to the model changed our accuracy noticeably.
lmao i literally ran into this same problem yesterday. the pace of papers is genuinely impossible to keep up with
not sure i agree with everything but synthetic data for training is the next big paradigm shift
curious how synthetic data helps with retrieval specifically though. generation quality maybe, but if your index is structured wrong or embeddings are mismatched, more training data doesn't fix that.