Implementation Experience: Retrieval-Augmented Generation (RAG) System for AI-powered Customer Support,A Senior Software Developer’s Perspective

In the rapidly evolving customer support landscape, integrating Artificial Intelligence (AI) has become increasingly pivotal. This essay elucidates the experiences and insights from implementing a Retrieval-Augmented Generation (RAG) system for AI-powered customer support, as observed by a senior software developer specializing in AI applications.
Implementing an AI chatbot utilizing the RAG system proved complex, surpassing the simplicity often portrayed in online tutorials. This project demanded a comprehensive approach, spanning several months of meticulous situation analysis and result evaluation. The success of such an endeavour hinges significantly on the expertise of a proficient DevOps, Software Engineer, and Data scientist team, which is essential for deploying the system across servers effectively.
While the RAG system did not achieve perfection in addressing all customer inquiries, its utility and value in resolving repetitive questions were undeniable. Moreover, it was a valuable asset to the marketing team, particularly when customers sought to gauge the company’s reliability through seemingly simple inquiries. The AI’s ability to provide prompt, consistent responses in these instances proved instrumental in bolstering customer confidence.

What would happen if a senior software developer with Artificial intelligence in the customer support team?

In this article, I want to share my experience making my first AI Chatbot with the RAG system. Unlike many YouTube videos, this work is not simple code and requires many steps. Understanding the situation and evaluating the results took at least 2–3 months. Also, a good DevOps team is needed to execute the whole system on servers. I want to say thanks to ali mh for their actions. In the end, the RAG system couldn’t answer all the questions and will not be perfect in many situations, but it is still useful and valuable and can solve repeated questions. And also it helps the marking team. Clients most of the time worry about their money, and they want to know about our company’s persistence. Sometimes, they ask a simple question to evaluate the support team and how reliable we are. In these cases, AI response can help a lot in answering their questions.

Response Quality

• 35% of responses were excellent, closely resembling those of human agents.
• 25% prompted requests for additional information, potentially expediting agent response times.
• 30% provided guidance, albeit with some deviation from the original request.
• 10% pertained to transaction tracking, requiring verification in our databases systems.

Making Dataset or FAQs

One of the important parts of every machine learning system is the dataset. An automatic system must gather all conversations between clients and agents, analyze them, and create datasets. Although making a dataset would be tough work, finding the most repeated questions and answers with the LLM model and storing them in a database is possible. Machine learning can help a lot in making data, but it doesn’t mean you can remove all human resources in the support team. After clustering and selecting frequency questions and answers, you must recheck them with humans and remove and modify the dataset. Most businesses want to hear that it takes only one time to make a dataset, but after investigation, we understand it should be done every week. One of the main reasons for this problem is that the application is not a static project. It could be changed, and guidance should be updated every week.
Furthermore, the mark will be changed, and new customers with new attitudes will join your business. Making a dataset will be a new job in new words. Client support: instead of directly answering all clients, they need to update FAQs and write total solutions.

Is the RAG system a total solution?

Unfortunately, not. In the best theory, the RAG system can solve repeated problems and offer solutions. If suddenly something bad happens to the application, it can’t answer clients well, and it is mandatory to update the dataset with humans. In addition, this system couldn’t track transition or find the money. For these situations, businesses need software to track transactions in the database and understand the situation.

What is a requirement to build RAG and auto-response systems?

• Gathering all questions and answers of the support team in the database.
• Clean up data and cluster and find related topics.
• Manually modify selected data.
• Make FAQs dataset.
• Using FAQs in RAG system.
• Evaluate result of RAG system.
• Update dataset with LLM and Manually
• Update and evaluate new models

What is the best LLM model?

Based on our experience, chatGPT-4o has the best result, and it is also necessary to use chatGPT-3.5 and llama to make a dataset.

In conclusion

Implementing a RAG system for AI-powered customer support represents a significant advancement in customer service capabilities. While not a panacea for all support scenarios, it offers substantial benefits in handling routine inquiries and enhancing customer perception of company reliability. The success of such a system relies heavily on continuous refinement, regular dataset updates, and a balanced integration with human expertise. As AI technology evolves, the potential for further improvements in customer support automation remains promising, heralding a new era in customer service efficiency and effectiveness.

Stackademic 🎓

Thank you for reading until the end. Before you go:

• Please consider clapping and following the writer! 👏
• Follow us X | LinkedIn | YouTube | Discord
• Visit our other platforms: In Plain English | CoFeed | Differ
• More content at Stackademic.com