Building a public-facing chatbot powered by Large Language Models (LLMs) comes with some unique challenges, ensuring responses are accurate, grounded, and aligned with ethical guidelines and business regulations.
In this session, we’ll explore how we leveraged Azure OpenAI to create a customer service chatbot that "plays nice"
Some key takeaways:
Grounding responses with confidence: How we leveraged external data sources, content moderation, and prompt engineering to ensure the chatbot consistently delivers accurate and grounded responses.
Preventing harmful interactions, techniques for mitigating risks of the chatbot engaging in inappropriate or harmful conversations, including ethical considerations.
Automating Evaluation Pipelines: How we developed automated systems to evaluate chatbot responses, ensuring measurable improvements and avoiding regressions when updating prompts or models.
Architectural decisions and Azure services that we used to create a scalable and resilient chatbot solution.
Lessons learned from real world usage
Robert has over 25 years experience of building Internet based solutions, including working as an architect at Microsoft, helping the first customers adopt Azure in the early days of cloud computing. As the Marketing Manager & Head of Cyber security at Active Solution he is responsible for the overall marketing strategy for the company. He is also responsible for driving the continuous work around cyber security and secure development. In customer projects he focuses on architecture for scalable cloud solutions, often involving cutting edge AI.
As a public speaker and a Microsoft Regional Director he does regular appearances at conferences like Developer Summit, Techorama and Oredev - speaking about AI, security, cloud computing and architecture - among other things.
Robert is also a hobby musician, whenever he finds the time he can be found noodling with old analog synthesizers and drum machines.
