Context Engineering for Large-Scale Product Search

Large language models are increasingly being used as components of complex agentic systems rather than as standalone conversational assistants. In this setting, the challenge shifts from designing prompts to designing context. Instead of asking "How should we phrase the instruction?", the question becomes "What information does the model need to accomplish the task?"

In this talk, I’ll show how we approached context engineering for product recommendations at Criteo, using an agentic shopping assistant as the example. I’ll walk through how we assemble the context in a fixed pipeline with two LLM calls, one that turns the chat into queries and filters and one that assesses each candidate's relevance to the user's intent. I’ll also discuss how we measure the results with an LLM judge calibrated on human labels. Then I’ll cover three levers for improving what the models see: enriching the product catalog, fine-tuning the product classifier, and finally fine-tuning a query-expansion model. That last part uses SFT on a frontier teacher followed by reinforcement learning on our own retrieval results. Along the way, we’ll look at when fine-tuning a lightweight model actually pays off: when latency, cost and serving control call for a small model with a narrow job.

Although the case study focuses on product search, the techniques apply broadly to agentic systems in which an LLM's primary role is not to generate the final answer, but to construct high-quality context for downstream components.