|
@@ -1,9 +1,13 @@
|
|
-Comparing Faiss with the original RAG. Using the TAIDE LLM model. Using created embeddings in the Supabase db.
|
|
|
|
|
|
+Comparing Faiss with the original RAG. Both using the TAIDE LLM model. Both using created embeddings from the Supabase db.
|
|
|
|
|
|
-Did not apply multi_query on Faiss (it will generate very similar answers from the original RAG). Applied multi_query on the original RAG only.
|
|
|
|
|
|
+Did not apply multi_query on Faiss (as it generates very similar answers from the original RAG). Applied multi_query on the original RAG only.
|
|
|
|
|
|
Evaluate on: 1. speed 2. RAGAS score
|
|
Evaluate on: 1. speed 2. RAGAS score
|
|
-Some of the result is in the output.txt
|
|
|
|
|
|
|
|
|
|
+Some of the tested result is in the output.txt
|
|
|
|
+
|
|
|
|
+Findings:
|
|
|
|
+- Response speed: It's not worth using Faiss to speed up llm on small dataset. Its index method is designed for large dataset.
|
|
|
|
+- Response quality: If relevant documents can be found, the original RAG which spplied the multi query method can generate more detailed answers. Faiss' answers are shorter.
|
|
|
|
|
|
Faiss: https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/
|
|
Faiss: https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/
|