Comparison of RAG Query Enhancement Methods: How to Choose Among Multi-Query, HyDE, and Step-Back
RAG query enhancement is a RAG query preprocessing method that rewrites or expands user questions before retrieval. This article compares Multi-Query, HyDE, and retrospective query rewriting to help evaluate retrieval for complex questions.
What Is RAG Query Enhancement?
RAG query enhancement is a class of methods that rewrite, expand, or abstract the user's original question before retrieval occurs in order to improve the quality of retrieval matches. RAG itself enhances answering capability by grounding large language model responses in external knowledge, but the final effect largely depends on whether the system can retrieve relevant and sufficient context.
In production-grade RAG systems, using only one original query is often not enough. Users may express themselves ambiguously, terminology in the knowledge base may be inconsistent, and a question may also require background concepts. Query preprocessing and query rewriting therefore become important entry points for improving retrieval effectiveness.
Multi-Query, HyDE, and Step-Back can be understood under the same type of problem: they do not directly generate the final answer, but instead change or supplement query clues before retrieval, giving the system a better chance of finding relevant documents. It should be noted that the current evidence directly supporting Step-Back is relatively weak. This article discusses it only as a framework for understanding “retrospective query rewriting” and does not expand on a fixed implementation process.
Key point: What RAG query enhancement solves is not the question of “how the model answers,” but the question of “what kind of query clues are given to the system before retrieval.”
How Do the Basic Ideas of the Three Methods Differ?
Multi-Query, HyDE, and Step-Back can all be discussed in the context of query enhancement in advanced RAG retrieval, but they supplement information in different directions. Multi-Query focuses on increasing expression coverage, while HyDE focuses on generating richer retrieval clues. In this article, Step-Back is only conservatively understood as an idea within retrospective query rewriting, leaning toward extracting higher-level or more contextualized query clues from a specific question.
| Method | Input rewriting approach | Information it aims to supplement | Observable applicable conditions | Retrieval cost | Main risks |
|---|---|---|---|---|---|
| Multi-Query multi-query expansion | Generate multiple query expressions around the same question | Synonymous expressions, different angles, different naming conventions | Unclear user expression, diverse keywords, inconsistent naming in the knowledge base | Usually increases, because multiple queries are needed | Recalls more irrelevant documents and increases downstream filtering pressure |
| HyDE hypothetical document embeddings | First generate a hypothetical document, then use its semantic representation to assist retrieval | Richer semantic context, expressions closer to document form | The original question is short, keywords are insufficient, or the question differs greatly from document wording | Increases generation and retrieval pipeline overhead | The hypothetical document may contain unreliable content and cannot be used as a factual answer |
| Step-Back retrospective query | Can be understood as generating higher-level or more contextualized query clues | May supplement higher-level concepts or background material | A candidate experimental strategy in complex question retrieval that needs result-based validation | Depends on whether additional retrieval is initiated | May introduce overly broad retrieval results, and the specific mechanism must be defined cautiously |
The three are not mutually exclusive. In real systems, one method can be selected according to the question type, or they can be combined in complex scenarios. However, combination strategies bring more retrieval attempts, latency, and debugging complexity, so they should not all be enabled by default.
Multi-Query Is Suitable for Improving Recall and Covering Different Expressions
The core practice of Multi-Query multi-query expansion is to generate multiple query expressions around the same user question and then initiate retrieval separately. It is suitable for cases where the “way the user asks” and the “way the knowledge base writes” are not completely consistent.
For example, if a user asks “How can I reduce the problem of RAG not finding materials,” the knowledge base may use different expressions such as “improve recall,” “query preprocessing,” “query expansion,” and “retrieval optimization.” A single query may only match some of them, while Multi-Query can expand coverage from multiple expression angles.
Applicable Scenarios
- The user's expression is unclear, and the original question lacks key terms.
- The same concept has multiple expressions, abbreviations, or naming conventions.
- The knowledge base has complex sources, and different documents use different expressions for the same topic.
- The priority is to improve recall first, and then use experiments to observe whether relevance improves at the same time.
Main Benefits and Costs
The direct benefit of Multi-Query is improving recall and covering different angles and expressions. For ambiguously expressed queries, it usually has a better chance than a single original query of matching relevant materials.
But its cost is also clear: multiple queries are required, so retrieval cost increases. If every expanded query enters a vector retrieval, keyword retrieval, or hybrid retrieval pipeline, system latency and resource consumption may also rise accordingly.
Usage suggestion: Multi-Query is more suitable as a candidate remedy for “insufficient recall,” rather than the default configuration for all questions. For simple factual queries, excessive expansion may instead introduce noise.
HyDE Supplements Retrieval Clues Through Hypothetical Documents
HyDE stands for Hypothetical Document Embeddings, usually translated as hypothetical document embeddings. Its basic idea is to first use an LLM to generate a “hypothetical document” based on the user question, and then use the semantic representation of this text to assist retrieval.
The hypothetical document here is not the final answer, but an intermediate retrieval artifact. Its role is to convert a very short, very abstract, or keyword-poor question into semantic clues containing more context.
Why Is HyDE Useful?
In many RAG scenarios, the user's question may be very short, such as “How should this error be handled?” or “How can recall be improved?” Such questions contain limited retrieval signals, while many knowledge base documents contain more context than the user's short question, such as background, causes, steps, or terminology.
HyDE attempts to supplement this semantic information by generating a hypothetical document. The system can then retrieve based on the embedding result of this text, potentially providing richer retrieval clues and thereby helping the system find relevant knowledge fragments.
Applicable Scenarios
| User need | Solution | Possible effect |
|---|---|---|
| The original question is too short and lacks keywords | Generate a hypothetical document containing more context | Increase semantic clues available for retrieval |
| The user's wording differs greatly from the knowledge base wording | Use the hypothetical document to transform it into an expression closer to documents | Help the system find relevant knowledge fragments |
| Background material needs to be retrieved from a conceptual question | First generate possible explanatory text, then retrieve related documents | Help recall more complete background content |
Misuses to Avoid
The text generated by HyDE may look like an answer, but it should not be directly treated as a factual conclusion. It is only an intermediate representation generated to help retrieval, and the final answer should still be built on retrieved external knowledge and the subsequent generation process.
Key point: HyDE's value lies in “improving retrieval clues,” not in “generating the answer in advance.” If the hypothetical document is directly output to the user, it confuses retrieval-assisting text with factual basis.
How Is Step-Back Conservatively Understood in This Article?
Within the current scope of evidence, this article only treats Step-Back as a framework for understanding retrospective query rewriting: it attempts to extract higher-level and more contextualized query clues from a specific question. It is not continuing to add synonymous queries, nor is it generating a hypothetical document; instead, it supplements retrieval with an entry point that may be closer to background material.
For example, a user may ask why a specific configuration does not take effect. Direct retrieval may only match a specific error or parameter description; the idea of retrospective query rewriting, however, may try to supplement relevant mechanisms or constraints, and then observe whether these background clues can help retrieve more useful documents.
It should be noted that different systems may implement this type of retrospective approach differently. In implementation, Step-Back should not be regarded as a fixed process, nor should it be assumed in advance that it is definitely suitable for a certain type of question. A safer approach is to validate it through retrieval results: does it really bring in more relevant background documents, or does it merely generate a broader question?
How Is It Different from Subquestion Decomposition?
| Method | Focus | Typical use |
|---|---|---|
| Subquestion query | Break a complex task into multiple smaller questions | Can be used for multi-step and multi-condition question handling |
| Step-Back retrospective query | Try to elevate a specific question into higher-level question clues | Can serve as a candidate idea for supplementary background retrieval |
Both can be classified as advanced query rewriting ideas, but they move in different directions. Subquestion decomposition leans more toward “horizontally breaking down the task,” while Step-Back leans more toward “looking upward for background.” This distinction is suitable for understanding the orientation of the methods, but the specific effect still needs to be experimentally validated against the knowledge base content and retrieval pipeline.
How Should It Be Validated?
If you want to try Step-Back or other retrospective query rewriting in a system, you can focus on the following questions:
- Whether the retrospective query still serves the original question rather than deviating from the topic.
- Whether the newly matched documents supplement necessary background instead of merely adding broad material.
- After keeping the original query results, whether the retrospective query can bring additional value.
- Whether the latency and cost introduced by additional retrieval are acceptable.
If the retrospective question is too abstract, the retrieval results may deviate from the original intent. Therefore, a safer approach is to retain the original query results and use the documents brought in by the retrospective query as supplements, rather than completely replacing the original retrieval.
How Should You Choose a Query Enhancement Method Based on the Scenario?
When choosing a RAG query enhancement method, the key is not to determine which one is “more advanced,” but to first identify the reason for retrieval failure. Different methods correspond to different retrieval difficulties.
| Retrieval problem | Method to test first | Reason for choice |
|---|---|---|
| The user's expression is ambiguous, with many synonyms | Multi-Query | Use multiple expressions to improve recall coverage |
| Knowledge base naming is inconsistent, and a single query gets few matches | Multi-Query | Trigger retrieval from different angles |
| The user's question is very short and lacks keywords | HyDE | Generate richer semantic clues |
| The user's question differs greatly from document expressions | HyDE | Use a hypothetical document to assist in finding relevant knowledge fragments |
| The question is too specific and may lack background material | Step-Back or other retrospective query rewriting | Can be used as a candidate solution for supplementary background retrieval and requires experimental validation |
| The question is relatively complex and the retrieval path is unclear | Subquestion query, Step-Back, or a combination strategy | Retrieval paths can be supplemented through decomposition or abstraction, but they should not be enabled by default |
In production systems, recall quality, the number of retrieval attempts, latency, cost, and stability must also be considered at the same time. Multi-Query may improve coverage, but it increases the cost of multiple queries; HyDE introduces an additional generation step; Step-Back or other retrospective query rewriting may bring background documents, but may also deviate from the original question due to excessive abstraction.
Selection principle: First identify the type of retrieval failure, then choose an enhancement strategy. Do not regard any method as a universal solution.
What Should Be Checked When Implementing RAG Query Enhancement?
Query enhancement changes the input of the entire RAG pipeline, so variables should be controlled as much as possible during evaluation. A safer approach is to first establish a baseline with the original query, then gradually add one enhancement method and observe whether the retrieval results truly improve.
Baseline and Experiments
- First keep a set of original query results as a control baseline.
- Introduce only one enhancement method at a time to avoid adding multiple variables simultaneously.
- Compare the matched documents before and after enhancement, rather than only checking whether the final answer is more fluent.
- Group questions by type, such as factual queries, conceptual queries, complex questions, and background-dependent questions.
Retrieval Quality
- Check whether the matched documents are more relevant to the user's question.
- Observe whether more necessary background is recalled, rather than more irrelevant content.
- If Multi-Query brings a large number of duplicate or low-relevance results, observe in experiments whether deduplication, ranking, or filtering is needed.
- Treat HyDE-generated content cautiously and do not regard it as a factual source.
- For Step-Back or other retrospective query rewriting, focus on checking whether the background documents still serve the original question.
Cost and Stability
- Record the additional number of retrieval attempts brought by the enhancement strategy.
- Pay attention to the latency and resource consumption introduced by multiple retrieval attempts.
- Keep simple factual queries lightweight; complex rewriting does not need to be enabled by default.
- Test stronger query rewriting or combination strategies for complex questions.
An executable implementation approach is: first use the original query to establish a baseline; if recall is insufficient, test Multi-Query; if semantic clues are insufficient or the difference between the question and document expressions is obvious, test HyDE; if background material needs to be supplemented, then test Step-Back or other retrospective query rewriting on a small scale. Finally, decide whether to launch based on retrieval relevance, latency, and cost.