Customer‑service chatbots
Also called AI customer support, AI chat agent, support bot, agent assist, AI helpdesk
AI that helps support staff has strong evidence; bots that replace them get mixed results and real liability. Evidence, costs and risks.
The best evidence favours AI that helps your staff, not AI that replaces them. Across 5,179 support agents, an assistant suggesting replies lifted issues resolved per hour by 14%, and by 34% for novices. When Alibaba let a staff-supervised AI agent answer shoppers directly, routine chats got faster but ratings fell. 64% of customers would prefer companies not use AI for service, and a tribunal held Air Canada liable for its chatbot's wrong answer.
At a glance
| Status | In pilots |
|---|---|
| What it does | Answers routine customer questions in web chat, email or WhatsApp, or suggests replies to human agents, and hands harder cases to a person. |
| Typical cost | Zendesk Suite Team from US$55 per agent per month billed annually, plus US$1.50 to US$2.00 per automated resolution beyond the included allowance (vendor list prices, checked Oct 2026). |
| Data it needs | Accurate, current policies and FAQs, 6 to 12 months of past tickets, and order or account data if it is to answer 'where is my order?' |
| Setup effort | Medium: the software is off the shelf, but writing and maintaining the knowledge it answers from is real work. |
| Evidence | Strong. Several large independent field studies, all in big firms: clear gains when AI assists agents, mixed results when it replaces them. |
What are customer-service chatbots?
Two quite different things are sold under this name.
- Customer-facing bots answer the customer directly on your website, by email or on WhatsApp. They resolve simple questions and pass the rest to a person.
- Agent-assist tools sit beside your staff. They read the conversation and suggest a reply, a help article or a next step, which the staff member can use or ignore.
Older bots followed scripted decision trees. Current tools use large language models that write their own answers from your help content. That makes them more flexible, and more able to say something wrong with confidence.
Use is widespread in large firms. The US Consumer Financial Protection Bureau estimated that about 37% of the US population used a bank's chatbot in 2022. We found no reliable figure for how many firms of 5 to 100 staff run one.
How does it work?
A modern support bot is a language model connected to a knowledge base: your FAQs, returns policy and delivery times and, ideally, live order data. When a customer asks a question, it finds the relevant text and writes an answer. If it is unsure, the customer is frustrated, or a rule says so, it hands over to a person.
Agent assist works the same way but shows its draft only to your staff member. In the best-known study, the tool was built to support agents rather than replace them, and agents were free to ignore its suggestions.
Pricing increasingly follows outcomes. Some help-desk vendors charge per conversation the AI resolves on its own, on top of per-agent seats, with a small allowance included. Check exactly how the vendor defines a resolution before you sign, because that definition drives the bill.
What it needs to work
A support bot is only as good as the knowledge it answers from and the escalation route behind it. Most failures trace back to stale policies, missing order data or no clean handover to a person.
- A written, current knowledge base covering returns, refunds, delivery times, warranty terms and pricing rules, with one named owner who updates it whenever a policy changes
- 6 to 12 months of past tickets or chats exported from Zendesk, Freshdesk, email or WhatsApp Business and tagged by topic, to find which questions are routine enough to automate
- Read access to order and delivery status in Shopify, WooCommerce or courier portals if the bot will answer 'where is my order?'
- A defined handover rule and a staffed human queue, with a target response time for escalated chats
- Baseline figures from before launch: first response time, resolution time, repeat contacts and customer satisfaction
- A written agreement with the vendor covering where chat logs are stored and who can access them
Most use cases in this guide succeed or fail here, on the data underneath them, before any AI is involved.
What does the evidence show?
The evidence is unusually good, and it splits cleanly.
AI assisting people works. In a study of 5,179 support agents at a Fortune 500 software firm, an AI assistant raised issues resolved per hour by 14% on average, and by 34% for novice and low-skilled agents. Customer sentiment improved and staff retention rose, and requests to speak to a manager fell by about 25%. The most experienced agents gained little and saw small declines in quality.
AI replacing people is mixed. In a randomised trial at Alibaba's Taobao covering 680,676 chats, an AI agent handling standardised queries cut chat time on those queries by about 17% but lowered customer ratings by 0.41 points on a five-point scale. Frustrated customers handed back to humans fared worst. In an earlier field experiment on outbound sales calls, revealing that the caller was a bot cut purchases by more than 79.7%.
AI assisting your staff vs AI replacing them
| Compared on | AI assists human agents | AI answers customers first |
|---|---|---|
| Best evidence | 14% more resolutions per hour; 34% for novices | Faster routine chats but lower ratings in a large trial |
| Customer reaction | Sentiment improved | 64% of customers prefer no AI in service |
| Who checks the answer | Your staff member, before it is sent | Nobody, unless it escalates; you remain liable |
| Cost model | Per-agent seat | Per-agent seats plus a fee per automated resolution |
| Best for | Training new staff, consistent answers, busy queues | High volumes of simple, standardised questions |
Status and evidence
We rate this pilots. The independent evidence is strong and large-scale, but it comes from big firms and splits. AI that assists agents shows clear gains; AI that replaces them lowered ratings in a randomised trial and meets customer resistance. We found no independent study of chatbots in businesses of 5 to 100 staff.
Where does it go wrong?
- Wrong answers you are liable for. Air Canada's chatbot told a bereaved customer he could claim a bereavement fare after travelling, which contradicted the airline's own policy page. The tribunal found negligent misrepresentation, held the airline responsible for everything on its website, chatbot included, and ordered it to pay C$812.02.
- Doom loops. The CFPB documented customers stuck in repetitive loops with no route to a person, including one who said they now faced a late fee as a result.
- Off-script behaviour. After a system update, DPD's bot swore at a customer and called DPD the worst delivery firm in the world; DPD switched off the AI element.
- Cutting too deep. Klarna's chief executive said cost had weighed too heavily and quality fell; the firm now promises customers can always reach a person.
- Customer resistance. 64% of customers would prefer companies not use AI for service, and 53% would consider switching.
- Stale knowledge. Change a returns policy without updating the knowledge base and the bot keeps promising the old one.
Numbers it should move
Ask the demo business
Northgate Supply is a fictional wholesale distributor of catering and packaging supplies: 28 staff, R46.6m in sales over the last year. Its data is generated for this demo, to 30 September 2026.
Half of September's 180 customer queries got a first reply within 2.2 hours, but 3 waited more than a working day. December was the slowest month, at 4.7 hours.
How this was worked out
Median hours from a customer query arriving to the first reply, per month, across email and WhatsApp.
Where the data usually lives
Where the records this use case needs are usually kept in a small business. It is not a list of recommended tools.
Frequently asked questions
How much does an AI customer service chatbot cost?
Help-desk vendors typically combine per-agent seats with a fee per AI resolution. Zendesk lists Suite Team at US$55 per agent per month billed annually, with automated resolutions at US$1.50 committed or US$2.00 pay-as-you-go beyond the included allowance (vendor list prices, checked Oct 2026). Add the staff time to write and maintain the knowledge base, which is often the larger ongoing cost.
Do I need a developer to set up a chatbot for my small business?
Not for a basic bot that answers from your FAQs; help-desk tools are set up through their own settings. You will need technical help if the bot must look up live orders, bookings or account details, because that means connecting it securely to your shop or other systems. Budget most of your time for writing accurate help content.
Am I liable if my chatbot gives a customer wrong information?
Very likely. In Moffatt v Air Canada, a Canadian tribunal rejected the airline's argument that it was not responsible for its chatbot, found negligent misrepresentation and ordered compensation. The US CFPB has warned that wrong chatbot answers may break laws requiring accurate information. Treat the bot's words as your own. This is general information, not legal advice.
Will customers be annoyed if I use an AI chatbot?
Many will. In a Gartner survey of 5,728 customers, 64% said they would prefer companies not use AI for service, and the top worry was difficulty reaching a person. Klarna moved to guarantee a human option after quality fell. Make the route to a person obvious and fast.
What do POPIA and UK GDPR require if a chatbot handles customer data?
Chat logs usually contain personal information. South Africa's POPIA requires you to tell people what you collect and why, keep it secure, and have a written contract with any operator, such as a chatbot vendor, that processes it for you. In the UK, the ICO's guidance on AI and data protection covers lawfulness, transparency, accuracy and impact assessments. This is not legal advice.
More in brand, marketing & customers
Sources
- 1Generative AI at WorkNational Bureau of Economic Research (Brynjolfsson, Li and Raymond; Quarterly Journal of Economics, 2025), 2023Study
- 2Generative AI at Work (NBER Working Paper 31161, full text)National Bureau of Economic Research, 2023Study
- 3Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba's Customer Service OperationsarXiv (Wang, Zhu, Feng, Lu and Jia), 2026Study
- 4Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer PurchasesMarketing Science, INFORMS (Luo, Tong, Fang and Qu), 2019Study
- 5
- 6Chatbots in consumer finance (Issue Spotlight)US Consumer Financial Protection Bureau, 2023Government
- 7Moffatt v. Air Canada, 2024 BCCRT 149Civil Resolution Tribunal of British Columbia, 2024Government
- 8
- 9
- 10Zendesk pricingZendesk, 2026Vendor
- 11Guidance on AI and data protectionInformation Commissioner's Office (UK), 2023Government
- 12Protection of Personal Information Act 4 of 2013Government of South Africa, 2013Government
This entry is educational. It is not legal, financial or professional advice, and it does not recommend any vendor. Figures attributed to vendors are their own claims.
- 1Generative AI at WorkNational Bureau of Economic Research (Brynjolfsson, Li and Raymond; Quarterly Journal of Economics, 2025), 2023Study
- 2Generative AI at Work (NBER Working Paper 31161, full text)National Bureau of Economic Research, 2023Study
- 3Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba's Customer Service OperationsarXiv (Wang, Zhu, Feng, Lu and Jia), 2026Study
- 4Frontiers: Machines vs. Humans: The Impact of Artificial Intelligence Chatbot Disclosure on Customer PurchasesMarketing Science, INFORMS (Luo, Tong, Fang and Qu), 2019Study
- 5
- 6Chatbots in consumer finance (Issue Spotlight)US Consumer Financial Protection Bureau, 2023Government
- 7Moffatt v. Air Canada, 2024 BCCRT 149Civil Resolution Tribunal of British Columbia, 2024Government
- 8
- 9
- 10Zendesk pricingZendesk, 2026Vendor
- 11Guidance on AI and data protectionInformation Commissioner's Office (UK), 2023Government
- 12Protection of Personal Information Act 4 of 2013Government of South Africa, 2013Government