.jpg)
The challenge
Bank Van Breda, a prominent Belgian bank, has a clear innovation roadmap. One of their top priorities in the coming year is optimizing their internal knowledge management system. This is where we, along with artificial intelligence, come into the picture.
Bank Van Breda, a prominent Belgian bank, has a clear innovation roadmap. One of their top priorities in the coming year is optimizing their internal knowledge management system. This is where we, along with artificial intelligence, come into the picture.
Their goal is twofold: to migrate their existing internal knowledge database to SharePoint and to establish an advanced question-and-answer (Q&A) system, a knowledge assistant, based on their internal knowledge. By doing this, the bank aims to enhance employee efficiency by enabling them to find information more quickly and easily.
As the organization was already in the process of migrating their existing internal knowledge database to SharePoint, we had the opportunity to advise them on structuring their new knowledge database so that an AI-powered knowledge assistant could optimally find and use the available information.
Our main goal was to develop an effective Q&A system — a knowledge assistant — completely tailored to our clients' needs. We started with a proof of concept (a customer care bot): a test to see if the migrated customer care files were adequately structured to feed the knowledge assistant with accurate information.
Before building our proof of concept, we evaluated several AI models (Large Language Models), ultimately selecting GPT-3.5 for its great balance of cost and quality. After selecting the AI model, we implemented our standard plug-and-play setup with Azure AI Search and Azure OpenAI Service. This resulted in the first version of the knowledge assistant that could search for and retrieve information from the documents in SharePoint and use this information to formulate an answer in natural language. This initial version was used to gather user feedback. These tests were crucial in identifying areas for improvement and ensuring the system met the necessary standards to create a knowledge assistant. Did the assistant already provide correct answers, or did it need some fine-tuning?
Based on user feedback, we iteratively started improving our assistant by fine-tuning the model (with custom embeddings) and prompt engineering. This customization enhanced the accuracy and relevance of search results, leading to higher-quality, context-specific answers.
We added a source verification mechanism to enhance the system’s reliability further. When the assistant formulates an answer, it always adds the sources on which the answer is based. This way, users can always double-check the source and whether the language model has cited it correctly.
Following the successful migration and pilot phase, our next step is to expand the knowledge assistant beyond customer care so every employee can use it. Moreover, we will train the bank’s technical teams through comprehensive training sessions to equip them with the necessary skills to independently maintain and enhance the system.
Want to know more?
