Ask‑your‑business chat
Also called Text-to-SQL, Chat with your data, Natural-language business intelligence, AI data assistant
Can AI answer questions from your own business data? What text-to-SQL benchmarks show, and how joined, defined data affects accuracy.
AI answers simple questions about clean data well; accuracy falls sharply on real company data and on arithmetic. On the public BIRD benchmark the best system scored 83%, with a hint per question, against 93% for data engineers and students. On real private data warehouses, a 2026 test found about 11%. On one firm's sales data, a model got 93% of simple totals right but 4% of arithmetic questions. A vendor of business-definition software reports written definitions lifted accuracy from about 46–51% to 68%.
At a glance
| Status | In pilots |
|---|---|
| What it does | Answers plain-English questions about sales, customers, debtors and costs by querying your own systems |
| Typical cost | Varies by vendor. Often bundled into accounting, CRM or office AI subscriptions; joining and defining data across several systems is usually quote-based |
| Data it needs | Joined, cleaned records from accounting, sales and CRM systems, plus written definitions of your key business terms |
| Setup effort | Low inside one system; High across several, because records must be matched and terms defined |
| Evidence | Moderate. Several independent academic benchmarks on accuracy; no independent study of business outcomes in small firms |
What is ask-your-business chat?
Ask-your-business chat lets an owner or manager type a question in plain English, such as which customers are paying more slowly than last quarter, and get an answer drawn from the company's own records rather than from the internet.
It comes in three common forms. Text-to-SQL tools turn the question into a database query and run it. AI features inside accounting, CRM or reporting software answer questions about the data that product already holds. And general AI assistants can be connected to business systems through connectors, increasingly using the open Model Context Protocol (MCP), so one chat window can reach the ledger, the CRM and the online shop.
All three promise answers without waiting for an analyst or building a report. All three depend on the same thing: whether the data underneath is complete, consistent and clearly defined.
How does it work?
The AI is given a description of your data: table names, column names and, ideally, written definitions of what terms such as active customer or gross margin mean in your business. It writes a query, runs it, and turns the result into a sentence or chart.
The hard part is not the language. It is mapping a casual question onto the right tables, joins and filters. Sales last month might mean invoiced or paid, with or without VAT, before or after credit notes. A person who knows the business resolves that without thinking. The model has to guess unless someone has written it down.
This is why the same AI can look excellent in a demo on a tidy sample database and unreliable on a real company's ledger, CRM export and stock spreadsheet sitting side by side.
What it needs to work
It works only as well as the data it reads. Results depend on whether your systems are joined, cleaned and defined consistently; pointing an AI at raw exports or spreadsheets gives fluent answers of unknown accuracy.
- At least 12 to 24 months of transactions in the accounting system, with sales, credit notes and cost of sales posted the same way every month
- Customer, product and supplier records matched across the ledger, CRM and online shop with a shared ID, so every system agrees who each customer is
- A written glossary of 20 to 50 business terms: what counts as revenue, an active customer, gross margin and overdue, and which date (invoice, delivery or payment) each figure uses
- One place where the joined data lives, such as a reporting database refreshed daily, rather than live access to raw exports and spreadsheets
- Read-only access limited to the tables the questions need, with payroll and other personal data excluded unless there is a clear reason
- About 20 test questions with known answers, rerun whenever the data, definitions or AI tool change
Most use cases in this guide succeed or fail here, on the data underneath them, before any AI is involved.
What does the evidence show?
The independent evidence is academic benchmarks. On BIRD, 95 databases across more than 37 domains, the top system scored 83.4% in October 2026, with a hint supplied per question, against 92.96% for data engineers and students. On Spider 2.0's enterprise-style databases, often over 1,000 columns, the authors' best agent solved 21.3% in 2024; its leaderboard's top entries in October 2026 report 76% on its Lite version and 97% on Snow. BEAVER, built from real private data warehouses, found about 11% in 2026, rising to 30.1% when the AI was handed expert hints for every step of each question.
On LG Electronics' internal sales data, one model got 93% of simple totals right but 4% of arithmetic questions such as ratios. Staff at a semantic-layer vendor, which sells software that stores agreed business definitions, report that a short document of business definitions raised accuracy on sample retail data from 46%–51% to about 68%.
Scoreboards are noisy: experts found annotation errors in 52.8% of BIRD Mini-Dev and 62.8% of Spider 2.0-Snow examples. We found no independent study of business outcomes from these tools in small firms.
Chat over joined, defined data vs chat over raw exports
| Compared on | Joined, defined data | Raw exports and spreadsheets |
|---|---|---|
| Simple totals (sales last month) | Usually right: 93% on one firm's sales database | Often right, but which version of sales? |
| Multi-step questions (ratios, rankings) | Better, but error-prone: with definitions, one vendor test still got about a third of all questions wrong | Unreliable, especially on multi-table sheets |
| Setup | Weeks: matching records, writing definitions, testing | Minutes |
| Error profile | Mostly wrong joins or filters on complex questions | Silent misreads of layouts, duplicates and subtotals |
| Best for | Recurring questions from managers who know the numbers well enough to spot an odd answer | A one-off look at a single clean table, checked by hand |
Status and evidence
Pilots: independent benchmarks show high accuracy on public test databases and poor accuracy on real private data warehouses, and we found no independent study of business outcomes in small firms. Evidence is moderate because several independent benchmarks agree on where it works and where it fails, although their own answer keys contain errors.
Where does it go wrong?
- Pointing it at raw exports. In a 2024 study of real spreadsheet-editing tasks from Excel forums, the best model then tested succeeded 18% of the time against 71% for Excel experts, and did worse on sheets with several tables or irregular headers.
- Undefined terms. If revenue means three different things across your systems, the AI picks one silently, and the answer looks equally confident either way.
- Unmatched records. ABC Trading in the CRM and ABC Trading (Pty) Ltd in the ledger are two customers to a machine, so customer rankings and debtor totals split.
- Multi-step questions. Ratios, differences and rankings fail far more often than simple totals.
- Trusting a tidy number. A wrong query still returns a clean answer. Check anything that drives a decision against a report you trust.
- Over-broad access. Assistants that also read email or documents can be steered by instructions hidden in that content; give them read-only access to what they need.
Numbers it should move
Ask the demo business
Northgate Supply is a fictional wholesale distributor of catering and packaging supplies: 28 staff, R46.6m in sales over the last year. Its data is generated for this demo, to 30 September 2026.
We invoiced R3.98m in September 2026, and R46.6m over the last 12 months. July to September came in 1.2% below April to June. The biggest month was December 2025, at R5.66m, on festive-season trade. Disposables is the largest category, at 22% of sales.
How this was worked out
Invoiced sales excluding VAT, after discounts, per calendar month, and each product category’s share of the 12-month total.
Where the data usually lives
Where the records this use case needs are usually kept in a small business. It is not a list of recommended tools.
Frequently asked questions
How accurate is AI at answering questions about my business data?
Good on simple totals over clean data, poor on arithmetic and messy data. On the public BIRD benchmark, the best system got about 83% of questions right in October 2026, with a hint supplied per question, against 93% for data engineers and students. On one company's own sales data, one model got 93% of simple totals right but only 4% of questions needing arithmetic, such as ratios. Treat anything beyond a simple total as a fast first draft, and check numbers that drive a decision.
Can I just upload my spreadsheets to an AI chatbot and ask questions?
For a quick look at one clean table, yes. For business questions it is risky. In a 2024 study of real spreadsheet-editing tasks from Excel forums, the best model then tested succeeded 18% of the time against 71% for Excel experts, and did worse on sheets with several tables or irregular headers. Merged cells, subtotals and inconsistent names cause errors you may not notice.
Why does the AI get my numbers wrong when the demo looked perfect?
Demos run on tidy sample data. Real data has duplicate customers, inconsistent product codes and terms that mean different things in different systems. A benchmark built from real private data warehouses found about 11% accuracy for the best systems tested, against over 80% on a public benchmark. Large firms report data problems too: in a 2026 survey of 505 data leaders at large firms, co-published by a data-software company, data quality ranked as the top challenge in seven of eight questions.
How much does an AI that answers questions from company data cost?
Asking questions inside software you already pay for is often bundled into its per-user or per-company subscription; prices vary by vendor. Asking across several systems costs more, mostly in data work: joining accounting, sales and stock systems, cleaning customer and product records, and writing definitions. One vendor of business-definition software found in its own study that written definitions lifted accuracy by 17 to 23 points on a sample retail dataset, though about a third of answers were still wrong.
Do I need a developer or IT team to set this up?
Not for the simplest form: asking questions inside the software that already holds the data. To ask across several systems, you need someone who can connect them, match records and maintain definitions, often an accountant who knows the ledger plus a technical contractor rather than a full-time team. In the BEAVER benchmark, giving the AI expert hints for every step of each question nearly tripled accuracy, though only to about 30%.
Is it legal under POPIA and UK GDPR to connect AI to my customer and staff data?
Both laws allow it when you meet their conditions. POPIA sets conditions for the lawful processing of personal information and is enforced by the Information Regulator. In the UK, the ICO's guidance on AI covers lawful basis, impact assessments, security and data minimisation. In practice, know which personal data the tool can reach, where it is processed, and whether your provider contract covers it. This is not legal advice.
More in leadership & reporting
Sources
- 1BIRD-SQL: A Big Bench for Large-Scale Database Grounded Text-to-SQLs (leaderboard)BIRD-SQL benchmark team, 2026Study
- 2Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL WorkflowsarXiv (ICLR 2025), 2024Study
- 3Spider 2.0 (benchmark leaderboard)Spider 2.0 benchmark team, 2026Study
- 4BEAVER: An Enterprise Benchmark for Text-to-SQLarXiv, 2026Study
- 5
- 6
- 7Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier ModelsCube, a semantic-layer software company (via arXiv), 2026Vendor
- 8
- 92026 State of Data Integrity and AI ReadinessPrecisely, a data-software company, with Drexel University LeBow College of Business, 2026Vendor
- 10LLM01:2025 Prompt InjectionOWASP Gen AI Security Project, 2025Commentary
- 11Information Regulator South Africa (home page)Information Regulator (South Africa), 2026Government
- 12Guidance on AI and data protectionInformation Commissioner's Office (ICO), 2023Government
This entry is educational. It is not legal, financial or professional advice, and it does not recommend any vendor. Figures attributed to vendors are their own claims.
- 1BIRD-SQL: A Big Bench for Large-Scale Database Grounded Text-to-SQLs (leaderboard)BIRD-SQL benchmark team, 2026Study
- 2Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL WorkflowsarXiv (ICLR 2025), 2024Study
- 3Spider 2.0 (benchmark leaderboard)Spider 2.0 benchmark team, 2026Study
- 4BEAVER: An Enterprise Benchmark for Text-to-SQLarXiv, 2026Study
- 5
- 6
- 7Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier ModelsCube, a semantic-layer software company (via arXiv), 2026Vendor
- 8
- 92026 State of Data Integrity and AI ReadinessPrecisely, a data-software company, with Drexel University LeBow College of Business, 2026Vendor
- 10LLM01:2025 Prompt InjectionOWASP Gen AI Security Project, 2025Commentary
- 11Information Regulator South Africa (home page)Information Regulator (South Africa), 2026Government
- 12Guidance on AI and data protectionInformation Commissioner's Office (ICO), 2023Government