Medibot: AI-Powered Medical Chatbot
A RAG-based medical chatbot with sub-200ms vector search.
Problem
Getting accurate answers to medical questions typically means digging through documentation manually, which is slow and easy to get wrong.
Challenge
Build a chatbot that retrieves relevant medical context fast enough to feel conversational, while keeping answers grounded in real source material instead of relying purely on a language model's memory.
Architecture
Medibot uses a retrieval-augmented generation (RAG) pipeline: incoming questions are embedded and matched against a FAISS vector index of medical reference content, the top matches are passed as context to a HuggingFace model through LangChain, and FastAPI serves the resulting answer.
Technical Decisions
FAISS for vector search
FAISS was chosen to keep retrieval latency under 200ms, since a RAG pipeline is only as responsive as its slowest step — and vector search is the step most likely to bottleneck as the reference corpus grows.
LangChain to orchestrate retrieval and generation
LangChain ties the retrieval step to the HuggingFace model call, keeping prompt construction and context injection in one place instead of scattered across the codebase.
FastAPI for the serving layer
FastAPI's async support and automatic request validation made it a straightforward fit for serving a latency-sensitive endpoint.
Implementation
- Built a FAISS vector index over medical reference content and tuned it for sub-200ms retrieval.
- Used LangChain to wire retrieved context into prompts sent to a HuggingFace model.
- Exposed the pipeline through a FastAPI endpoint.
- Iterated on chunking and retrieval parameters to raise context-aware accuracy to 85%.
Results
- Cut query resolution time by 60% compared to manual lookup.
- Achieved sub-200ms latency on FAISS-based vector search.
- Reached 85% context-aware accuracy in responses.
Lessons Learned
- Most of the perceived intelligence in a RAG system comes from retrieval quality, not the language model itself — tuning chunking and embeddings mattered more than prompt tweaking.
- Keeping vector search fast is what makes a RAG pipeline feel conversational rather than sluggish.
Future Improvements
- Add source citations to responses for transparency.
- Expand the reference corpus and re-evaluate retrieval accuracy at scale.