In this guide · 7 sections
Short answer
On SEA-HELM's Malay board (18 Sep 2026, open models up to 200B), Qwen 3.5, Gemma 4, Qwen 3.6 and AI Singapore's SEA-LION v4.5 were tied at the top, around 75 to 76. For ChatGPT, Claude or Gemini, test with your own content and get a native Malay speaker to review.
What SEA-HELM shows for Malay
SEA-HELM is AI Singapore's benchmark for Southeast Asian languages. Its Malay score combines cultural knowledge, instruction following, general knowledge, multi-turn chat, generation (NLG), understanding (NLU) and safety tasks in Malay. The snapshot below is the leaderboard's default view on 18 September 2026: open-weights models up to 200 billion parameters.
| Model | Maker | Malay score | 95% range |
|---|---|---|---|
| Qwen 3.5 (122B) | Alibaba | 76.31 | -1.72 / +1.80 |
| Gemma 4 (31B) | 76.27 | -1.70 / +1.77 | |
| Qwen 3.6 (27B) | Alibaba | 76.26 | -1.76 / +1.72 |
| SEA-LION v4.5 (Qwen, 27B) | AI Singapore | 75.23 | -1.73 / +1.67 |
| Muse Glimmer (30B) | Meta | 74.14 | -1.83 / +1.79 |
| SEA-LION v4.8 (Nemotron, 120B) | AI Singapore | 73.25 | -1.71 / +1.74 |
| GPT OSS (120B) | OpenAI | 69.68 | -1.82 / +1.98 |
| Mistral Medium 3.5 (128B) | Mistral AI | 68.93 | -1.70 / +1.66 |
The top four are within each other's confidence ranges, so treat them as tied. SEA-HELM says it also evaluates 16 closed-weights models, such as those behind ChatGPT, Claude and Gemini, through their APIs; the public view we read lists open-weights models. Our SEA-LION guide covers the leaderboard in more detail.
Where models are weakest in Malay
The breakdown matters more than the headline. On the same 18 September board, the top eight models all scored highest on Malay generation and lowest on safety and cultural tasks.
| Competency | Range across the top 8 models | What it means for you |
|---|---|---|
| NLG (generation) | 88.67 to 93.09 | Fluent Malay sentences are the easy part |
| Instruction following | 80.25 to 90.72 | Mostly follows format and length rules |
| Cultural | 33.04 to 71.14 | Local customs, festivals and references need checking |
| Safety | 34.39 to 46.14 | Sensitive topics in Malay are handled less reliably |
For a business, that translates into a rule: AI Malay will usually read smoothly, but anything touching religion, Hari Raya greetings, food certification, or community sensitivities needs a careful human check.
Where machine Malay goes wrong for Singapore readers
Malay and Indonesian share a lot, and AI models have seen far more Indonesian text online. The result is Malay with Indonesian words mixed in, which Singapore readers spot immediately.
| Problem | Example | Fix |
|---|---|---|
| Indonesian vocabulary | mobil instead of kereta (car) | Ask for Singapore or Malaysian Malay, not Indonesian |
| False friends | percuma means free in Malay but in vain in Indonesian | Have a native speaker check promotional words |
| Overly formal register | Textbook or government-style Malay in a casual post | Specify the tone: friendly, everyday |
| Literal translation | English idioms translated word for word | Ask for a fresh Malay version, not a translation |
| Local names and places | Wrong spellings of estates, MRT stations or mosques | Supply the correct names in the prompt |
Specify the audience in every prompt: "Bahasa Melayu for Singapore readers, not Indonesian". It doesn't guarantee clean output, so a reviewer still needs to check.
SEA-LION or a mainstream AI tool?
AI Singapore's SEA-LION models score close to the best open models in Malay and are free to download. That doesn't make them the easiest choice for a small business.
Use SEA-LION if
- You want a model you can host on your own servers
- You are building a chatbot or app that must handle Malay and other regional languages
- You have a developer to set it up
Use ChatGPT, Claude or Gemini if
- You just need Malay drafts for posts, notices and emails
- You already pay for a business AI plan
- You want the simplest workflow for staff
For everyday content, test whichever business AI plan you already use against your own Malay examples before buying anything new. If you are also producing Chinese content, our guide to the best AI for Chinese-English translation covers that language pair.
How to run your own 30-minute Malay test
Since no public benchmark tests Singapore Malay marketing copy across the mainstream tools, a quick test with your own material is the best guide.
Pick three real pieces
For example a promotion post, an outlet notice and a short product description.
Write one prompt with clear audience instructions
Bahasa Melayu for Singapore readers, not Indonesian; friendly tone; include your exact prices in S$, dates and place names.
Run it in two or three tools
Use your business AI plans with identical wording; label outputs A, B and C.
Ask a native Singapore Malay speaker to score them
Naturalness, Indonesian words, accuracy against the brief, and whether it sounds local.
Save the winning prompt
Reuse it as a template and keep a short list of words the reviewer corrected.
Do it yourself or get help?
For posts and notices, AI drafts plus a review by a native Malay speaker on your team are usually enough. For ads, signage and anything printed, build in that review every time.
Our Extra Ad Creative Pack (S$290) delivers 6 static ad designs from your photos, each exported in 1:1, 4:5 and 9:16, with 3 headline and 3 primary-text variants, checked against platform text and image rules, editable source files and 2 rounds of revisions. For Malay versions, send us your approved Malay text in the brief. IT Sorted is a Singapore-based team of developers with over 10 years of experience; you pay upfront and brief us in writing.
Frequently asked questions
On AI Singapore's SEA-HELM Malay leaderboard dated 18 September 2026, looking at open-weights models up to 200B, Qwen 3.5, Gemma 4, Qwen 3.6 and SEA-LION v4.5 (Qwen) scored between 75 and 77, within each other's confidence ranges. Closed models such as ChatGPT, Claude and Gemini are not in that public view, so test them with your own content.
ChatGPT can write fluent Malay, but we found no current public Malay benchmark covering it. The common problem with AI Malay generally is Indonesian vocabulary slipping in, such as mobil instead of kereta. Ask for Malay for Singapore readers, not Indonesian, and have a native speaker check anything customers will see.
SEA-HELM, the Southeast Asian Holistic Evaluation of Language Models, is AI Singapore's benchmark for Southeast Asian languages, including Malay, Indonesian, Thai, Tamil, Vietnamese, Filipino and Burmese. It scores models on chat, instruction following, knowledge, cultural understanding, generation, understanding and safety tasks in each language, and publishes results on a public leaderboard.
Only if you need a model you can host yourself or are building a regional-language chatbot with a developer. SEA-LION v4.5 (Qwen) scored 75.23 on SEA-HELM's Malay board, close to the leaders, and is free to download. For everyday Malay posts and notices, a business AI plan you already use is simpler; test it first.
Malay and Indonesian are closely related, and AI models have seen much more Indonesian text online, so they often mix in Indonesian words and spellings. Some words even change meaning: percuma means free in Malay but in vain in Indonesian. Tell the tool the audience is Singapore Malay readers and get a native speaker to review.