Hi, I'm Aaditya!
I'm a Senior Software Engineer with over 12 years of experience in search and information retrieval. I've built search platforms serving tens of millions of users, evolving them from traditional lexical approaches to modern hybrid architectures that combine keyword matching with semantic retrieval and LLM-powered understanding. Making search work at that scale means solving hard distributed systems problems: real-time pipelines processing millions of events daily, zero-downtime migrations across hundreds of billions of records, and infrastructure tuned for sub-second latency. Outside of work, I research novel retrieval pruning techniques aimed at making RAG systems faster and more cost-effective.
Papers
Abstract
Hybrid retrieval, which fuses a sparse lexical retriever with a dense neural one, is widely recommended in the software documentation surveyed in September 2026, usually with reciprocal rank fusion (RRF), on the strength of aggregate gains. We ask what fusion costs on individual queries, treating “do not fuse” as a first-class outcome, in the setting in which hybrid retrieval is deployed: a standard Anserini BM25, five dense encoders spanning 22M–596M parameters and three model generations, a learned-sparse (SPLADE) alternative, and a cross-encoder second stage, on seven BEIR datasets and three fusion functions. Even the best method, dense-weighted score interpolation, ranks relevant documents below the better of its two components on 13–48% of queries and below the deployable dense-only retriever on 8–21% (14–48% and 11–25% at a weight selected under leave-one-corpus-out validation), while beating dense-only in aggregate on every corpus with the primary encoder; the downside is significant under loss-averse risk-sensitive evaluation on six of the seven corpora. We call this loss the fusion tax. It is mostly a ranking effect that preserves recall in aggregate, it is most likely on the queries that only one retriever covers, and it persists across the encoder ladder and the SPLADE substitution. RRF carries the higher tax wherever the dense retriever is clearly stronger and only beats dense-only where the two components are balanced. Fusion failure is weakly predictable within a corpus but transfers unreliably across corpora, because the features that predict it change sign with the sparse–dense balance. A cross-encoder recovers part of the tax only where it is itself reliable. We distill these findings into guidance that keys the fusion decision to the measured sparse–dense balance of the target corpus.
BibTeX
@article{chauhan2026fusiontax,
author = {Chauhan, Aaditya and Agarwal, Arun},
title = {The Fusion Tax: A Diagnostic Analysis of When Hybrid
Retrieval Hurts},
journal = {IEEE Access},
volume = {14},
pages = {159620--159637},
year = {2026},
month = oct,
doi = {10.1109/ACCESS.2026.3743502},
issn = {2169-3536}
}
Abstract
We describe our system for LitTraceQA (GroundLM @ EMNLP 2026): given a research question, retrieve the relevant papers from a pool of 27,487, cite the page and the table or figure where the answer lives, and answer in a requested format. Our main finding is that evidence grounding is limited by retrieval, not by reading. The page selector put the annotator's page, which we call the gold page, in front of the model that locates evidence only about half the time (52.6% gold-page recall), while that model, given the page, cited the right one in 45 of the 48 locators it emitted (94%). When the page was missing it rarely said so: of 45 such cases it returned nothing 14 times, a wrong page 24 times, and a correct page 7 times, so the pipeline failed quietly almost twice as often as it failed visibly. Since the failure was that the right page was never shown, the fix is to stop choosing: each retrieved paper fits in the model's context, so we show it whole. Page ranking survives only as a fallback inside papers too long to fit, which no test-split paper was, and gold-page recall reaches 100% on the papers we can parse. Separately, questions that identify their target by position rather than content, such as “the first author of the 24th reference”, are served by parsing rather than retrieval: we resolve the bibliography into an addressable list, which also supplies identifiers the evidence metric scores. The final system scores 0.762 paper F₁, 0.441 evidence F₁ and 0.920 multiple-choice accuracy on the held-out test split. Because the pipeline depends on a closed model without seed control, we release a harness that verifies the paper's central claims against committed artifacts.
BibTeX
@misc{chauhan2026pagerecall,
author = {Chauhan, Aaditya},
title = {PageRecall: Measuring Page Selection in Literature-Grounded
Question Answering},
year = {2026},
eprint = {2609.18154},
archivePrefix = {arXiv},
primaryClass = {cs.IR}
}
The Limits of Behavioral Pruning: Why Enterprise RAG Lifecycle Management Is a Governance Problem
2026Abstract
Enterprise RAG systems accumulate embedding corpora that grow indefinitely, yet lack principled lifecycle management for their vector indexes. We propose behavioral pruning—using access patterns and semantic redundancy to select which documents to retain in the active index—centered on a governance mechanism we call semantic anchors: documents with no near-duplicates above a similarity threshold are preserved in the active index regardless of access frequency, and near-duplicate clusters retain their newest member, providing a traceable justification for retention decisions. We present an empirical evaluation identifying both conditions where the framework succeeds and where it fails. The recall advantage of behavioral pruning is conditional on corpus characteristics: random pruning achieves higher recall on historical queries (0.118 ± 0.011) than all structured methods; density-weighted behavioral pruning improves recall on fresh queries by 12% over semantic-only pruning. A supplementary simulation on BEIR benchmarks confirms behavioral signals help on redundant corpora with correlated access; on diverse corpora with uncorrelated access, recall degrades by up to 31%.
BibTeX
@inproceedings{chauhan2026limits,
author = {Chauhan, Aaditya},
title = {The Limits of Behavioral Pruning: Why Enterprise RAG Lifecycle
Management Is a Governance Problem},
booktitle = {Proceedings of the 28th International Conference on Enterprise
Information Systems (ICEIS 2026) - Volume 1},
pages = {338--345},
year = {2026},
publisher = {SCITEPRESS},
doi = {10.5220/0014912100004018},
isbn = {978-989-758-834-1},
issn = {2184-4992}
}
Abstract
The RAG community devotes disproportionate attention to optimizing text chunking strategy. Through a factorial experiment crossing four chunking strategies, two embedding models, two chunk sizes, and two QA datasets (Natural Questions and MS-MARCO), we find that, in QA-style retrieval on two established datasets, choice of dataset (24–78% of variance) and embedding model (11–50%) together explain 67–89% of retrieval quality variance, while chunking strategy accounts for only 1–2% and is not statistically significant (p > 0.21). An end-to-end LLM-as-judge evaluation confirms that chunking differences do not propagate to generation quality in this QA setting (Kruskal-Wallis p > 0.88). We argue that, for QA-style retrieval on general-purpose natural-language corpora, chunking strategy is a low-variance factor, and that RAG benchmarks should report variance decomposition alongside absolute metrics so practitioners can allocate engineering effort to the factors that actually drive performance.
BibTeX
@inproceedings{chauhan2026chunking,
author = {Chauhan, Aaditya and Hegde, Nivas},
title = {Chunking Is Not the Bottleneck: Why RAG Evaluation Must Look
beyond Segmentation Strategy},
booktitle = {Proceedings of the 15th International Conference on Data Science,
Technology and Applications (DATA 2026) - Volume 1},
pages = {605--612},
year = {2026},
publisher = {SCITEPRESS},
doi = {10.5220/0015150400004091},
isbn = {978-989-758-854-9},
issn = {2184-285X}
}
Talks
Abstract
Moving AI agents from prototypes to production requires more than a powerful language model. Enterprise systems must orchestrate agents and tools, manage context, enforce security and tenant isolation, integrate with existing services, and ground responses in complex organizational knowledge.
The session explores these challenges through two complementary perspectives. The first presents a working model inspired by an enterprise customer service scenario, demonstrating architectural patterns for enterprise Copilot systems: multi-tenant design, multi-agent and tool orchestration, security, graphical workflow design, and direct MCP-based tool invocation.
The second focuses on keeping agents grounded so they answer from real data instead of hallucinating. Following one customer question end to end, it covers identity files that set an agent’s rules and boundaries, MCP-connected knowledge bases, and the search layer underneath: why vector search alone misses exact names and terms, how hybrid retrieval fixes that, how multimodal embeddings reach knowledge in tables and images, and the compute, memory and cost walls retrieval hits as knowledge bases grow to billions of documents.
Writing
Every embedding model gets deprecated eventually. A practitioner's guide to migrating a production RAG pipeline without breaking search quality or your budget.
After years building search systems at hundreds of billions of records, here's what I learned about scaling vector search from kNN to HNSW to DiskANN.
A deep dive into the algorithms and engineering tradeoffs behind building a people-name search system at production scale.
Exploring whether a CS degree still makes sense in an era where AI is reshaping the software engineering profession.
Discussing the limitations of vector-only RAG pipelines and how combining BM25 with vector search via Reciprocal Rank Fusion enhances retrieval quality in production.
Resume
Technical Expertise
Search, AI & RAG: OpenSearch, Elasticsearch, hybrid search (BM25 + dense vector), semantic retrieval, Retrieval-Augmented Generation (RAG) pipelines, LLM integration, prompt engineering, NER-based query understanding, relevance tuning
Distributed Systems: Real-time event-driven architectures (Kinesis, SQS, Lambda, Step Functions), shadow testing frameworks, high-availability design
Data & Infrastructure: DynamoDB (GSI strategies, transaction handling), streaming pipelines, monitoring & operational excellence
Languages & Tools: Java, Python, JavaScript, AWS (OpenSearch, DynamoDB, Kinesis, Lambda, SQS, Step Functions, S3, CloudWatch, IAM), Git
Experience
Senior Software Engineer, Amazon
12+ years building enterprise-scale search and distributed systems- Led the evolution of search platforms from lexical to hybrid architectures combining BM25 with dense vector retrieval, serving tens of millions of users
- Designed and implemented RAG pipelines connecting search infrastructure with large language models for context-aware information retrieval
- Architected real-time event-driven data pipelines processing millions of events daily with strict consistency guarantees
- Built Named Entity Recognition systems and prompt engineering pipelines for query understanding, significantly improving search relevance
- Led multiple zero-downtime data migration projects across hundreds of billions of records with comprehensive shadow testing frameworks
- Mentored engineers through complex system re-architectures, led cross-functional design reviews, and established operational best practices
Projects
TabTime
2026A privacy-first productivity dashboard for Chrome. Combines time tracking, multi-timers, rich-text notes, daily scheduling, and stock monitoring, all with local-only data storage.
Webtime Tracker
2025The original Firefox version of the productivity suite. Features hydration reminders, calendar heatmaps, and local-only data storage.
Education
M.S. in Computer Science & Engineering, The Ohio State University
B.E. in Electronics and Communication, Netaji Subhas Institute of Technology, New Delhi