From hallucinations to a robust AI chatbot for e-commerce

Written by Tobias Riis Christensen, March 2025

When we first started working with a generative AI chatbot nearly three years ago, the technology showed promise but was unreliable. Early bots were often inaccurate and sometimes gave answers that looked correct but were just plain wrong. Instead of discouraging us, these early challenges helped us understand both the potential and the limits of generative AI.

Over the next few years, we moved carefully from experimentation to implementation, driven by a clear vision: to develop a chatbot capable of genuinely enhancing customer experiences for the 250 webshops relying on our service.

Finding the right partner

Choosing the right technology partner was our first big challenge. We needed a provider who understood how complex our operations are, could smoothly integrate with our existing systems, and – most importantly – shared our goal of improving customer service through automation. Digital Genius met these needs and shared our approach, setting the stage for a strong partnership.

Transforming knowledge into training

For the chatbot to be truly useful, it needed to match the quality and accuracy of our human customer service agents. This required well-structured, reliable training data – something we were lucky to already have in our detailed shop manuals. We used these manuals by marking content specifically for chatbot use with curly brackets { }. This simple but effective method allowed us to:

• Dynamically create relevant help centre articles
• Develop an API for instant access to client-specific context
• Keep the knowledge base updated automatically to ensure accuracy

Even with good data, accuracy was still a challenge. To solve this, we developed a careful five-step process to generate responses, making sure each reply was accurate, personalised, and in line with the client’s brand guidelines.

The result was a chatbot that felt truly human and matched each brand’s voice naturally.

Here is the process:

1. We analyse the customer message to fully understand the query
2. A separate AI system estimates a general answer
3. We search our knowledge base for the five most similar articles based on the question and estimated answer
4. A clear and concise reply is created using insights from these articles
5. The reply is adjusted to align with the client’s specified tone of voice and the specific question the customer asked
   

This process can be slightly slower at times, but most people agree it is better to wait a few extra seconds for the right answer – and it is still faster than a human response.

Smooth integrations and thoughtful limits

Integrating the chatbot smoothly into our existing environment posed several important considerations.

Our internal platform, ExportStation, is already linked seamlessly with more than 40 e-commerce systems via standardised APIs, simplifying the process significantly. However, integration was not merely about technical feasibility; it also involved thoughtful decisions on how extensively we wanted the chatbot to engage customers without human intervention.

"Where is my order?" – the most common customer question

This is by far the most frequent query we receive. At first, the chatbot could manage it easily if the customer had their order number. But what if they did not? This made us think carefully about the balance between helpful automation and making things too complicated.

How we solve it today

Currently, we address this issue by asking customers for their email addresses and looking up their most recent orders. To verify their identity securely, we send a verification code directly to their email. If this fails or the customer encounters an issue, we swiftly route them to a human agent.

This approach works well, but we keep asking ourselves whether we should go further:

• Should we expand verification methods, perhaps by adding SMS verification for phone numbers, giving customers more options to confirm their identity?
• Could we make things even easier by linking both phone and email to retrieve all past orders, allowing the chatbot to provide more detailed answers?
• Or would these changes add complexity without significantly improving the customer experience?

We are tracking how this process performs, but we are also continuously assessing when automation stops being helpful and starts feeling excessive. Understanding this balance has become a key part of our ongoing integration strategy.

Our goal remains clear: make things more helpful, not just more capable for the sake of it.

Looking ahead

Now that our generative AI chatbot is live, we are already seeing positive results: more automation, higher customer satisfaction, and smoother operations for our clients. But this is just the beginning. With ongoing monitoring, we will stay flexible and ready to refine and improve based on real-world performance and feedback.

That is where things stand for now! I will be back in a few months to share an update on how it is working in practice and what comes next. If you would like to try our version of the chatbot, you can take it for a spin here.