This article provides a brief summary of the paper on "HybridRAG," a method designed to make RAG systems more resistant to technical jargon. The original paper, titled HybridRAG: Enhancing RAG with Knowledge Graphs, was proposed by researchers from BlackRock and NVIDIA in August 2024. You can find the paper here.
HybridRAG was developed to address the weaknesses of the typical RAG (Retrieval Augmented Generation) system that relies on "vector search," improving the accuracy of answers. Vector search, while effective in many cases, has several limitations, such as poor handling of technical terminology and an inability to fully grasp context when answering questions.
The Weaknesses of Vector Search
Vector search has some known weaknesses, particularly in technical areas:
- Poor Handling of Technical Terms: It struggles with the precise understanding of technical jargon.
- Weak Contextual Understanding: When creating answers, vector search often passes only small segments of documents (e.g., 200 characters) to the large language model (LLM), which can result in insufficient context comprehension for generating accurate answers. This is problematic for complex tasks like answering questions based on financial statements, where understanding the entire document is crucial.
HybridRAG: Combining VectorRAG and GraphRAG
HybridRAG is a system that simultaneously uses VectorRAG (vector search) and GraphRAG (knowledge graph search) to retrieve relevant information. By combining the results from both methods, HybridRAG addresses the weaknesses of each, leading to more accurate responses.
Methodology
-
Preparation:
- Extract the text from the documents to be used for RAG (the paper uses financial documents as an example).
- Split the extracted text into chunks and vectorize them for VectorRAG.
- Construct a knowledge graph from the documents for GraphRAG.
-
Query Processing:
- VectorRAG searches for relevant document chunks based on the user's question.
- GraphRAG searches for relevant subgraphs from the knowledge graph.
- The results from both searches are combined and passed to the LLM, which generates the final answer.
The structure of the HybridRAG method is relatively simple. If you have already built a vector search-based RAG system, you can implement this by adding GraphRAG, which is another advantage of this approach.
Results
HybridRAG showed significant improvements in answering domain-specific and technical questions, particularly in the financial industry. The table below shows some key performance metrics:
- Faithfulness: 0.96 (VectorRAG: 0.94, GraphRAG: 0.96)
- Answer Relevancy: 0.96 (VectorRAG: 0.91, GraphRAG: 0.89)
- Context Recall: 1.00 (VectorRAG: 1.00, GraphRAG: 0.85)
(Note: The context precision (CP) is slightly lower than traditional methods because both search results are packaged into the context for final processing.)
Key Takeaways:
- HybridRAG accurately answered complex and domain-specific questions, particularly those involving technical terminology.
- It demonstrated high performance in both extraction tasks (which are weak points for vector search) and abstract question answering (a weakness of GraphRAG).
HybridRAG offers a powerful solution for domains where precise terminology and deep contextual understanding are crucial, making it a valuable tool for industries such as finance and beyond.