We had a rag based bot, which used user shared documents for returning appropriate responses. The issue here for every query the whole flow was triggered, even for repeated questions. Our approach here was to introduce a caching and retraining layer so that we can identify similar or same queries and return response from the cache.
Key Impact Takeaways:
- This activity will reduce the cost of GPT by around 40%
- With the setup of telemetry, we will get good insights of what kind of answers are being looked for so that we can tune our bot accordingly.
- This project is part of the sunbird DPG ecosystem and is currently used in multiple government and private forums.
- This feature will improve the response time by 70-80% for frequently asked questions.
- Contributor Renuka is continuing on as a full-time developer with organization Tekdi Technologies.