Voice call analysis in 1 cent. Review 20-minute calls in 20 seconds with Generative AI

Posted on : July 19th 2024

Can you use the latest AI technology (Generative AI) to analyze customer support conversations and provide actionable insights?

The short answer is YES!

We recently analyzed ~20 call audio files and transcribed, analyzed and audited them as an experiment to understand the topics discussed, determine if issues were resolved, gauge customer satisfaction, and ensure CSRs followed Standard Operating Procedures (SOPs).

Here’s a screenshot of the demo that hosts all calls, summarized and analyzed on the parameters of customer service representative’s behavior, customer’s problem solving and more.

You can play the demo here.

The immense potential of AI in customer support

To give you a bit of clarity on how we landed on this experiment, here’s a story.

In early 2019, a major customer support provider with 250,000 seats across 17 global centers faced a persistent challenge. Despite their efforts, their Customer Satisfaction (CSAT) scores peaked between 80 to 85.

More than 60% of their support was voice-based, spread across 27 languages. The client’s request was straightforward yet ambitious: use the latest AI technology to analyze customer support conversations and provide actionable insights.

With the client's commitment to improving CSAT, even a 1% lift could significantly impact their business metrics.

We began by transcribing all calls using cloud-based services, eventually selecting Google’s transcription service for its accuracy.

Identifying issues from transcripts was streamlined using the client’s master list of issues. However, gauging customer happiness proved challenging, achieving only a 70-73% accuracy.

The project culminated in an MVP demo but was shelved due to high processing costs of $23 per call.

With advancements in Generative AI, we revisited and revamped our solution.

Experimenting with models like Llama-3, Claude 3 Haiku, Gemini 1.5, and GPT-4, we landed on Claude 3 Haiku for its performance and cost-effectiveness.

• Transcription: We transcribe calls using the open-source Whisper model, significantly reducing operating costs.
• Speaker Identification: We use Anthropic's Claude 3 Haiku model to label speakers based on context. A schema helps achieve accurate diarization.
• Issue Analysis: Using structured prompts, we list customer issues, resolutions, reasoning, and next steps. This generates a clear, actionable table.
• Agent Evaluation: The model evaluates CSRs on several parameters, such as politeness, rapport, empathy, and the need for apologies. Detailed feedback highlights strengths and improvement areas.

Business Benefits this Experiment Highlights for Customer Service Teams

• How long does it take? Analyzing a 20-minute call takes less than 30 seconds.
• How much does it cost? Using Claude 3 Haiku costs under 1 cent per call. More powerful models like GPT-4o could cost up to 20 cents.
• Can this run privately? Yes. You can run this in your data center or on a gaming laptop using open-weight models like Whisper and LLama-3.

Interesting? If you want to learn more on how Generative AI can help you achieve good CSAT scores and enhance CX, book a discovery call with us.