Quality Coach
An AI-powered real-time coaching system built inside Customer Service Suite, helping support agents improve grammar, tone, and clarity while actively replying to customers.
My role
I worked on designing the end-to-end experience for Quality Coach, an AI-powered assistance layer integrated directly within the agent reply composer.
The project focused on helping support agents maintain high-quality customer interactions in real time without disrupting their workflow. I collaborated closely with Product, AI, Engineering and CX stakeholders to simplify how AI-generated feedback could be surfaced contextually while keeping suggestions actionable rather than overwhelming.
A major part of the challenge involved balancing proactive AI assistance with conversational flow so agents still felt in control of responses.
Audited after the fact. Coached too late.
Customer support teams often struggle to maintain consistent service quality across large volumes of conversations, especially during high-pressure support periods. Traditional QA systems primarily relied on post-conversation audits where feedback was shared after interactions had already concluded, creating several operational gaps:
- 01Agents repeated avoidable mistakes
- 02Quality feedback arrived too late
- 03QA teams spent excessive time auditing manually
- 04Service quality varied significantly between agents
- 05Managers lacked real-time visibility into quality
- 06Coaching loops never closed during live shifts
Support teams needed a proactive quality system capable of helping agents improve conversations while they were actively responding to customers.
Three questions,
one north star.
- How might we help support agents improve conversation quality in real time without disrupting active customer interactions?
- How might we reduce dependency on post-conversation audits through proactive AI assistance?
- How might we surface contextual AI feedback without overwhelming agents cognitively during live conversations?
Why traditional QA kept missing the moment.
Before designing Quality Coach, I wanted to understand how support teams currently handled quality assurance and where operational inefficiencies emerged inside existing workflows. The research focused on agent behaviour during live conversations, post-conversation QA processes, feedback fatigue, AI-assisted writing behaviour and real-time coaching expectations.
Traditional QA was reactive
Most quality systems evaluated conversations after they ended. Audits surfaced patterns but rarely helped agents during the moments where mistakes actually happened.
Agents struggled at peak volume
Tone, grammar and compliance slipped while handling multiple chats at once. The issue wasn't knowledge, it was cognitive overload.
Too much feedback created friction
Agents quickly ignored systems that generated excessive or intrusive suggestions. Concise, contextual feedback outperformed long-form correction.
Managers lacked live visibility
QA managers could only spot patterns after conversations finished. Teams needed live operational visibility, not retrospective analysis alone.
AI had to feel collaborative
Agents preferred assistive AI over autonomous AI. They wanted suggestions while still owning the final response, a foundational principle for the design.
Key takeaways
- 01Quality improvement needed to happen during conversations, not after
- 02Lightweight contextual suggestions beat intrusive AI interventions
- 03Cognitive overload was a bigger problem than lack of training
- 04Real-time coaching meaningfully reduced repetitive QA effort
- 05AI worked best when agents kept response ownership
Agents didn't want another reviewer.
Through workflow analysis, stakeholder discussions and support interaction reviews, I observed clear behavioural patterns across agents and QA teams. Agents focused heavily on speed and resolution, unintentionally overlooking tone consistency, grammar quality or conversational clarity during busy periods. QA managers, on the other hand, spent significant time manually auditing conversations after completion while recurring quality issues kept repeating across interactions.
Agents didn't want another tool to review them. They wanted intelligent assistance that helped them respond better without interrupting their flow.
This became the foundation for embedding Quality Coach directly inside the reply composer instead of creating a separate review interface.
Research objectives
- Understand operational gaps in traditional QA workflows
- Identify moments where agents commonly make mistakes
- Reduce cognitive overload during conversations
- Explore how AI suggestions could feel contextual instead of intrusive
- Improve conversation quality without increasing handling time
Key findings
- Real-time guidance was significantly more valuable than post-conversation feedback
- Concise suggestions performed better than long-form AI corrections
- Agents preferred contextual inline recommendations
- QA managers wanted operational visibility into coaching effectiveness
- Conversation quality improved when AI feedback was actionable and immediate
An assistance layer, not an audit tool.
Quality Coach introduced a proactive AI assistance layer directly within the agent reply experience. The system continuously evaluated replies in real time and surfaced contextual quality suggestions across seven dimensions, either applied instantly or ignored entirely. Agents stayed in control while still benefiting from AI-powered coaching.
- 01Grammar
- 02Tone
- 03Sentence length
- 04Profanity
- 05Relevancy
- 06Filler words
- 07Spelling
- PlusAlways on
A coaching layer no one else built.
I analyzed conversational quality and AI assistance experiences across Intercom, Zendesk, Salesforce Service Cloud, Grammarly Business and Kore.ai. Most platforms either focused heavily on post-conversation QA reviews or provided generic writing assistance disconnected from customer support workflows. Very few systems offered deeply contextual conversational coaching directly embedded inside the support reply experience.
Key market insights
Support teams were shifting toward AI-assisted operations
CX organizations increasingly expected AI to proactively improve agent productivity and quality, not just function as passive analytics.
QA teams were overwhelmed by manual audits
Traditional QA workflows scaled poorly as conversation volumes increased, putting operational pressure on quality managers.
Real-time assistance had stronger behavioural impact
Immediate feedback created higher correction rates compared to delayed audit-based coaching, across every cohort we studied.
AI systems needed to remain assistive
Support agents responded more positively to collaborative AI that enhanced responses instead of replacing them entirely.
Our edge
- Real-time contextual AI coaching
- Inline conversational quality analysis
- Lightweight assistive suggestions
- Actionable feedback over generic scoring
- Embedded composer experience
Mapping coaching opportunities.
We mapped coaching opportunities using an Impact / Effort framework to identify which AI interventions created the highest improvement in conversation quality without disrupting agents during live interactions.
- Inline grammar correction
- Filler-word reduction
- Real-time tone and profanity analysis
- Contextual relevancy engine
- Extended analysis
- Partial resolve
- Auto-rewriting full replies
- Mandatory AI approval gating
Key takeaways
- 01Lightweight inline coaching created significantly better adoption than disruptive review-based systems
- 02Contextual suggestions improved agent confidence without increasing cognitive overload
- 03Real-time intervention reduced repetitive quality issues before conversations reached QA audits
What we built.
Real-time quality coaching
An AI-powered coaching layer that continuously analyzed agent responses during active customer conversations and surfaced immediate quality recommendations, so agents could improve responses before messages were sent.
Contextual inline suggestions
Suggestions for tone, grammar, profanity, sentence length, relevancy and spelling appeared directly within the reply composer, reducing workflow interruption.
Actionable AI corrections
Every suggestion could be applied in one tap or ignored entirely, so agents fixed issues without leaving the composer and always kept ownership of the reply.
QA visibility & analytics
Managers could track coaching adoption, accepted and ignored suggestions, and CSAT and quality scores with and without Quality Coach, without waiting for audits.
The numbers.
reduction in repeated grammar, tone & profanity issues across agents
drop in time spent by QA teams auditing routine conversations manually
lift in CSAT score during high-volume shifts where Quality Coach was active
of agents kept Quality Coach enabled after the first two-week trial
Analytics showed customer sentiment stayed closer to positive at all times when agents used Quality Coach to handle conversations
Beyond the metrics, Quality Coach established the foundation for Customer Service Suite's broader AI copilot vision, combining real-time coaching, conversational intelligence and QA visibility within one assistive layer.
A product is never finished.
Quality Coach was designed as an early foundational layer for AI-assisted customer support experiences inside Customer Service Suite. The long-term vision extends beyond grammar and tone correction toward fully contextual conversational intelligence, capable of proactively improving resolution quality, compliance adherence, empathy and operational efficiency during customer interactions.
Future iterations focus on deeper AI personalization, smarter contextual understanding, workflow-aware suggestions and tighter integration with broader AI copilot capabilities across the support ecosystem.
