Daphné Vermeiren
This is a Raccoons-MbarQ collaboration. 🤝
In the previous parts of The Generative AI Playground, we discussed various generative AI applications on their own. This article focuses on the synergies created when different AI models, both ‘traditional’ machine learning and generative AI, are integrated to work together. Collaborating with our wonderful partners at MbarQ, we developed a demo for nexxworks by leveraging our collective generative AI experience. The goal? Envisioning a customer journey of the future for the energy sector.
Our starting point was to envision a potential customer journey within the energy sector. We started by identifying moments where AI could enhance or automate processes or steps to deliver a more streamlined experience to clients of energy providers. Ultimately, the scenario we thought out revolved around optimizing the upselling process in the energy sector, for example for solar panels.
For this experiment, we want to integrate various AI systems to support a sales representative who’s able to keep a helicopter view over the sales process at all times, while being supported by automated actions. To do this, we identified seven steps in the customer journey where artificial intelligence could play a role.
⚠️ Disclaimer: It’s crucial to underline that this scenario was designed as a thought experiment to push the boundaries of what’s currently being achieved with the technology. We have no intention of implementing the system in production as is, especially when looking at the concerns about privacy and the desirability of such pervasive surveillance in everyday life. This demo only serves to spark discussion and inspire innovation by illustrating the possibilities of artificial intelligence. It’s about showing its potential, not only in the future, but even today, as it’s already technically feasible to build every step we will be describing in this article.
The journey begins with an advanced machine learning (AI) algorithm to analyze energy consumption data. By processing energy consumption datasets, this system is able to identify patterns indicative of inefficient energy use, which might suggest an opportunity for solar panels. Such a system operates by comparing current consumption data against historical norms and flags potential leads who could benefit from an energy consultation. This step already shows how important ‘traditional’ machine learning algorithms are important to combine with newer generative models.
To illustrate this step, imagine a guy named Jim. Jim is super happy as the first real days of summer finally arrived in Belgium — it took nature long enough — and for Jim, sun means warming up his pool, putting on the barbecue and throwing an amazing pool party. Normally, Jim’s energy consumption is comparable to that of a typical consumer, but once the sun comes out, it spikes. And that’s exactly what the AI algorithm detects.
Once potential leads are identified, a second system comes into play, utilizing satellite images to visually determine whether the lead’s property already has solar panels installed. This is done by a sophisticated computer vision algorithm, trained to recognize installations from aerial photos. This step not only confirms the presence or absence of solar panels but can also assess the condition and layout of existing installations, providing valuable data without the need for physical site visits.
Back to Jim. Jim likes to be sustainable, and after renovating his house, he’s been saving up for solar panels. However, he lost track of it for a while… So, the AI algorithm does not detect solar panels on his roof.
The AI-generated insights of steps 1 and 2 are then sent to a sales representative who performs a human validation step. We like to call this a “human in the loop” system: the sales representative assesses the AI’s findings, considers additional factors like customer demographics, to make a calculated decision on whether to engage the customer further. This approach ensures a high level of accuracy and relevance in the sales cycle.
Jan looks at the AI’s insights. Jim’s case file is the personification of her target audience. Perfect. Let’s move on!
Customers who pass the initial screening get a call. An AI-driven conversational system autonomously conducts a comprehensive intake interview, asking detailed questions about:
By using a combination of a large language model (e.g., OpenAI’s GPT-4o) for generating human-like text responses, OpenAI’s Whisper for speech-to-text capabilities, and OpenAI’s text-to-speech and voice cloning, we can mimic a natural and engaging dialogue with the depth of a human sales agent. Moreover, this all occurs over a regular phone call. We use Twilio to facilitate the telephone conversation and integrate all AI functionalities seamlessly. This approach ensures that the user does not need to install any special apps or navigate to a web application but simply receives a phone call which is fully transcribed automatically in the background.
Jim gets a call. He doesn’t really like calling — like many millennials— but he decides to pick up. On the other side, the AI agent starts the conversation and explains why it’s calling: Jim’s energy consumption seems to have spiked and they noticed he does not have solar panels yet. Maybe they can help him get the best offer? Jim smiles. He’s interested — he was planning to purchase them eventually — and engages in the conversation.
This is not your typical, exhausting ‘robot asks questions one-by-one and user answers’ phone call. It’s a dynamic conversation where both parties can steer the dialogue. Jim can jump from one topic to another, circle back to information given three questions ago, or introduce new details the robot initially didn’t ask for. Everything flows naturally. This kind of flexibility even extends to language preferences, allowing Jim to request to continue the conversation in a different language if he wishes to.
After a 5-minute conversation, the AI agent has everything it needs to get the best quote for Jim. The AI agent now knows he has a swimming pool, aircon, and he has a south-oriented roof of approximately 40m2.
After gathering detailed information from the intake interview, another system structures the raw data into a coherent, structured format that can be easily analyzed, written away to the CRM, and acted upon (similar to our 6th article in the playground).
We now move to a second phase: AI-supported contacting of solar panel providers supplier. The system automatically generates requests for quotes which are sent out to partnered solar panel providers. This step not only speeds up the process but also reduces errors associated with manual data entry. Here, we could add another human validation step before the requests for quotes are sent, as you can see below.
Jim answered every question needed to generate the perfect request for quotes. The AI system, based on all the information at its disposition, now autonomously generates a Request For Information to be sent to solar panel providers. The sales representative, Jan glances over the request for a minute and decides it’s okay to send out to their solar panel partners. Sent!
After the request for proposals is sent, another AI system takes over to analyze the incoming proposals received from vendors. The system is able to scan the sales representatives’ mailbox over the coming weeks and link the correct proposal to the correct case file. Then, it autonomously starts to analyze the proposals received from vendors. It is able to compare various offers based on factors like price, warranty, provider reliability and it is even able to conduct initial negotiations with vendors to refine the terms where possible. Finally, it selects the top proposals based on predefined criteria, preparing a detailed comparison to aid the customer in making an informed decision.
Once again, the sales representative is in the loop. The representative can glance over the analysis of the offers and the communication that is about to be sent to the customer. Once agreed, a mail is sent out automatically to the customer via a mailbox connection. Note that this is a deliberate omni-channel set-up. While the first contact with the customer was conducted over the phone, now the customer receives an email, which is a better option for communicating offers.
In the following two weeks, the AI system finds 5 proposals in the sales representative's mailbox. The system automatically uploads them in the CRM and starts analyzing them. Ultimately, Jan gets a notification, highlighting the three best proposals, ready to be forwarded to Jim. Perfect. She adds a personal touch to her generated email, ensuring Jim he can always call her if he should have any questions. Jim receives the email, but as he’s already behind the barbecue of his pool party, he’ll only be able to check it the next morning…
Of course, the implications of this system extend beyond this demo — we want to show you how such technologies can be applied across various industries to (partially) automate complex workflows. For example:
Have any other use case in mind? We’re all ears!
Setting up this experiment has proven that we can already automate complex workflows with the technology that exists today. Moreover, we’ve learned that it’s important to:
Most importantly, the key to these applications is not full automation but strategic integration where AI handles routine and data-intensive tasks, allowing humans to focus on areas requiring judgment, creativity, and personal touch. This approach not only enhances efficiency but also elevates the roles of human employees, allowing them to engage more meaningfully in their work.
If you’re interested in experiencing the full demo, simply send us a message via hello@raccoons or our contact form and we’ll show you the entire journey — phone call included. And for now… on to the next experiment!
Written by
Daphné Vermeiren
Want to know more?
