Which AI Model for a Customer Service Chatbot? Cost vs Quality

Small, mid-range or frontier: what a support chatbot needs

AI vendors sell each model family in sizes. Small models are fast and cheap, mid-range models handle more complex reasoning, and frontier models top the benchmarks at a much higher price. A typical small-business chatbot answers questions from a known set of facts: opening hours, prices, delivery areas, booking rules, return policy. That is a reading-and-summarising job, which small models now do well when they are given the right documents.

Price per 1,000 replies, by model

Prices below come from each vendor's API price list: OpenAI, Anthropic and Google. The per-reply figures assume a round illustration of 2,000 input tokens (instructions, FAQ extract and chat history) and 200 output tokens per reply, with no caching. Our guide to LLM tokens and cost per customer reply works through the maths in detail.

Even the mid-range option costs about US$6 for 1,000 replies. For most small businesses, model choice is a quality and speed decision, not a budget one. Our API price comparison covers more models.

Reply speed: why reasoning settings matter for chat

Many current models can think before answering, and vendors offer settings from none or low up to max. Higher settings improve benchmark scores but can delay the first word dramatically. A customer waiting in a chat window notices anything beyond a few seconds.

The pattern is clear on the Artificial Analysis leaderboard: the same model can answer in one second or one minute depending on the setting. For a chatbot, test models at their lowest setting first and only raise it if answers are wrong.

Tool use and accuracy: what published evals tell you

A support bot often needs to call tools: check an order, look up a booking slot, hand over to a human. Artificial Analysis runs the tau-squared bench telecom test, a simulated technical support conversation where agent and customer both take actions. On its tau-squared bench leaderboard, the top scores have reached about 99%, and the test is listed among its legacy evaluations. In other words, simple tool calling is largely solved; differences show up in harder, longer conversations.

For FAQ accuracy, the biggest factor is not the model but what you give it. A small model reading your actual price list will beat a frontier model guessing from general knowledge. Ground the bot in your own content, tell it to say "I don't know" and hand over when the answer is not there, and keep the knowledge base up to date.

Languages and tone for Singapore customers

Singapore customers may write in English, Singlish, Chinese, Malay or Tamil, sometimes mixing them in one message. Published leaderboards do not test this mix well, so it is worth checking yourself. Larger models generally handle code-switching and less common languages more reliably, which can justify a mid-range model if many enquiries are not in English.

How to run your own 30-minute model test

Benchmarks narrow the field; your own questions decide. This test needs only API access or a provider playground.

1. 1

   Collect 20 real customer questions

   Take them from WhatsApp, email or your contact form, including a few awkward ones, a complaint and two in other languages.

2. 2

   Write one system prompt with your FAQ

   Include prices, hours, policies and the rule to hand over when unsure. Use the same prompt for every model.

3. 3

   Run two or three models at low settings

   For example GPT-6 Luna, Claude Haiku 5.5 and Claude Sonnet 5.5. Note how long each takes to start answering.

4. 4

   Score each answer

   Correct, partly correct or wrong, plus whether it made anything up. Count hand-overs that should and should not have happened.

5. 5

   Pick the cheapest model that passes

   If two models tie, choose the faster one. Re-run the test when you change the FAQ or when vendors release new versions.

Do it yourself or get help?

Off-the-shelf chatbot platforms let you choose a model from a drop-down. The work that makes a bot useful is the knowledge base, the guard-rails and the hand-over to a person. Our guide to AI chatbots for websites compares platforms and costs.

Our AI Website Assistant (S$790) sets up a chatbot trained on your website, FAQ and documents, with lead capture, hand-over to WhatsApp or email, English plus one more language and guard-rails so it only answers about your business. AI usage fees are billed to you directly.