Systex/faiss_index

Comparing Faiss with the original RAG.

Setup:

• Both using the TAIDE LLM model.
• Both using pre-created embeddings from the Supabase db.
• Did not apply multi_query on Faiss (it generates very similar answers as the original RAG). Applied multi_query on the original RAG only.

Findings:

• Response speed: Faiss is faster but the answers are shorter. Probably because it did not use multi_query. It's not really worth using Faiss to speed up llm on small dataset as its index algorithm is designed for quick search in large dataset.
• Response quality: If relevant documents can be found, the original RAG (which spplied multi query) can generate more detailed answers. Faiss' answers are shorter compare to the original RAG.
  
• Some of the tested result is in the output.txt. Using the reviewed QA as ground truth.

Run the code:

• Run ./run.sh to prepare the environment and download the Taide model.
• Run /faiss_index/faiss_index.py

Faiss: https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/