This post is co-written with Mayur Patel, Nick Koenig, and Karthik Jetti from GoDaddy.
GoDaddy empowers everyday entrepreneurs by providing all the help and tools to succeed online. With 21 million customers worldwide, GoDaddy’s global solutions help seamlessly connect entrepreneurs’ identity and presence with commerce, leading to profitable growth. At GoDaddy, we take pride in being a data-driven company. Our relentless pursuit of valuable insights from data fuels our business decisions and works to achieve customer satisfaction.
In this post, we discuss how GoDaddy’s Care & Services team, in close collaboration with the AWS GenAI Labs team, built Lighthouse—a generative AI solution powered by Amazon Bedrock. Amazon Bedrock is a fully managed service that makes foundation models (FMs) from leading AI startups and Amazon available through an API, so you can choose from a wide range of FMs to find the model that is best suited for your use case. With the Amazon Bedrock serverless experience, you can get started quickly, privately customize FMs with your own data, and integrate and deploy them into your applications using the AWS tools without having to manage infrastructure. With Amazon Bedrock, GoDaddy’s Lighthouse mines insights from customer care interactions using crafted prompts to identify top call drivers and reduce friction points in customers’ product and website experiences, leading to improved customer experience.
GoDaddy’s business challenge
Data has always been a competitive advantage for GoDaddy, as has the Care & Services team . We realize the potential to derive meaningful insights from this data and identify key call drivers and pain points. In the world before generative AI, however, the technology for mining insights from unstructured data was computationally expensive and challenging to operationalize.
Solution overview
This changed with GoDaddy Lighthouse, a generative AI-powered interactions analytics solution, which unlocks the rich mine of insights sitting within our customer care transcript data. Fed by customer care interactions data, it enables scale for deep and actionable analysis, allowing us to:
- Detect and size customer friction points in our product and website experiences, leading to improvements in customer experience (CX) and retention
- Improve customer care operations, including quality assurance and routing optimization, leading to improvements in CX and operational expenditure (OpEx)
- Deprecate our reliance on costly vendor solutions for voice analytics
The following diagram illustrates the high-level business workflow of Lighthouse.
GoDaddy Lighthouse is an insights solution powered by large language models (LLMs) that allows prompt engineers throughout the company to craft, manage, and evaluate prompts using a portal where they can interact with an LLM of their choice. By engineering prompts that run against an LLM, we can systematically derive powerful and standardized insights across text-based data. Product subject matter experts use the Lighthouse platform UI to test and iterate on generative AI prompts that produce tailored insights about a Care & Services interaction.
The below diagram shows the iterative process of creating and strengthening the prompts.
After the prompts are tested and confirmed to work as intended, they are deployed into production, where they are scaled across thousands of interactions. Then, the insights produced for each interaction are aggregated and visualized in dashboards and other analytical tools. Additionally, Lighthouse lets GoDaddy users craft one-time generative AI prompts to reveal rich insights for a highly specific customer scenario.
Let’s dive into how the Lighthouse architecture and features support users in generating insights. The following diagram illustrates the Lighthouse architecture on AWS.
The Lighthouse UI is powered by data generated from Amazon Bedrock LLM calls on thousands of transcripts, utilizing a library of prompts from GoDaddy’s internal prompt catalog. The UI facilitates the selection of LLM model based on the user’s choice, making the solution independent of one model. These LLM calls are processed sequentially using Amazon EMR and Amazon EMR Serverless. The seamless integration of backend data into the UI is facilitated by Amazon API Gateway and Amazon Lambdas functions, while the UI/UX is supported by AWS Fargate and Elastic Load Balancing to maintain high availability. For data storage and retrieval, Lighthouse employs a combination of Amazon DynamoDB, Amazon Simple Storage Service (Amazon S3), and Amazon Athena. Visual data analysis and representation are achieved through dashboards built on Tableau and Amazon QuickSight.
Prompt evaluation
Lighthouse offers a unique proposition by allowing users to evaluate their one-time generative AI prompts using an LLM of their choice. This feature empowers users to write a new one-time prompt specifically for evaluation purposes. Lighthouse processes this new prompt using the actual transcript and response from a previous LLM call.
This capability is particularly valuable for users seeking to refine their prompts through multiple iterations. By iteratively adjusting and evaluating their prompts, users can progressively enhance and solidify the effectiveness of their queries. This iterative refinement process makes sure that users can achieve the highest-quality outputs tailored to their specific needs.
The flexibility and precision offered by this feature make Lighthouse an indispensable tool for anyone trying to optimize their interactions with LLMs, fostering continuous improvement and innovation in prompt engineering.
The following screenshot illustrates how Lighthouse lets users validate the accuracy of the model response with an evaluation prompt
After a prompt is evaluated for quality and the user is satisfied with the results, the prompt can be promoted into the prompt catalog.
Response summarization
After the user submits their prompt, Lighthouse processes this prompt against each available transcript, generating a series of responses. The user can then view the generated responses for that query on a dedicated page. This page serves as a valuable resource, allowing users to review the responses in detail and even download them into an Excel sheet for further analysis.
However, the sheer volume of responses can sometimes make this task overwhelming. To address this, Lighthouse offers a feature that allows users to pass these responses through a new prompt for summarization. This functionality enables users to obtain concise, single-line summaries of the responses, significantly simplifying the review process and enhancing efficiency.
The following screenshot shows an example of the prompt with which Lighthouse lets users meta-analyze all responses into one, reducing the time needed to review each response individually.
With this summarization tool, users can quickly distill large sets of data into easily digestible insights, streamlining their workflow and making Lighthouse an indispensable tool for data analysis and decision-making.
Insights
Lighthouse generates valuable insights, providing a deeper understanding of key focus areas, opportunities for improvement, and strategic directions. With these insights, GoDaddy can make informed, strategic decisions that enhance operational efficiency and drive revenue growth.
The following screenshot is an example of the dashboard based on insights generated by Lighthouse, showing the distribution of categories in each insight.
Through Lighthouse, we analyzed the distribution of root causes and intents across the vast number of daily calls handled by GoDaddy agents. This analysis identified the most frequent causes of escalations and factors most likely to lead to customer dissatisfaction.
Business value and impact
To date (as of the time of writing), Lighthouse has generated 15 new insights. Most notably, the team used insights from Lighthouse to quantify the impact and cost of the friction within the current process, enabling them to prioritize necessary improvements across multiple departments. This strategic approach led to a streamlined password reset process, reducing support contacts related to the password reset process and shortening resolution times, ultimately providing significant cost savings.
Other insights leading to improvements to the GoDaddy business include:
- The discovery of call routing flows suboptimal to profit per interaction
- Understanding the root cause of repeat contact interactions
Conclusion
GoDaddy’s Lighthouse, powered by Amazon Bedrock, represents a transformative leap in using generative AI to unlock the value hidden within unstructured customer interaction data. By scaling deep analysis and generating actionable insights, Lighthouse empowers GoDaddy to enhance customer experiences, optimize operations, and drive business growth. As a testament to its success, Lighthouse has already delivered financial and operational improvements, solidifying GoDaddy’s position as a data-driven leader in the industry.
About the Authors
Mayur Patel is a Director, Software Development in the Data & Analytics (DnA) team at GoDaddy, specializing in data engineering and AI-driven solutions. With nearly 20 years of experience in engineering, architecture, and leadership, he has designed and implemented innovative solutions to improve business processes, reduce costs, and increase revenue. His work has enabled companies to achieve their highest potential through data. Passionate about leveraging data and AI, he aims to create solutions that delight customers, enhance operational efficiency, and optimize costs. Outside of his professional life, he enjoys reading, hiking, DIY projects, and exploring new technologies.
Nick Koenig is a Senior Director of Data Analytics and has worked across GoDaddy building data solutions for the last 10 years. His first job at GoDaddy included listening to calls and finding trends, so he’s particularly proud to be involved in building an AI solution for this a decade later.
Karthik Jetti is a Senior Data Engineer in the Data & Analytics organization at GoDaddy. With more than 12 years of experience in engineering and architecture in data technologies, AI, and cloud platforms, he has produced data to support advanced analytics and AI initiatives. His work drives strategy and innovation, focusing on revenue generation and improving efficiency.
Ranjit Rajan is a Principal GenAI Lab Solutions Architect with AWS. Ranjit works with AWS customers to help them design and build data and analytics applications in the cloud.
Satveer Khurpa is a Senior Solutions Architect on the GenAI Labs team at Amazon Web Services. In this role, he uses his expertise in cloud-based architectures to develop innovative generative AI solutions for clients across diverse industries. Satveer’s deep understanding of generative AI technologies allows him to design scalable, secure, and responsible applications that unlock new business opportunities and drive tangible value.
Richa Gupta is a Solutions Architect at Amazon Web Services specializing in generative AI and AI/ML designs. She helps customers implement scalable, cloud-based solutions to use advanced AI technologies and drive business growth. She has also presented generative AI use cases in AWS Summits. Prior to joining AWS, she worked in the capacity of a software engineer and solutions architect, building solutions for large telecom operators.
Source link
lol
