In this guide · 8 sections
Short answer
Multimodal models can read photos, scanned documents, PDFs and audio as well as text. They work well for receipts, forms and summaries, but make mistakes with small, rotated or handwritten text, charts and counting. Check outputs, stay within file limits, and treat photos of people as personal data.
What 'multimodal' means in plain English
Early chatbots only read and wrote text. Multimodal models turn images, document pages and audio into the same kind of internal representation as text, so you can ask questions about a photo or a PDF in ordinary language. Our plain-English LLM explainer covers how the text side works.
In practice, 'multimodal' covers three input types a business uses most: images (photos, screenshots, scans), documents (PDFs, Word files) and audio (voice notes, call recordings). Output is usually text, although some models can also produce images or speech.
Practical business uses
| Input | Example job | Watch out for |
|---|---|---|
| Receipt or invoice photo | Extract supplier, date, amount and GST into a spreadsheet | Faded thermal receipts, handwritten amounts |
| Site or defect photos | Describe defects and draft a site report | Precise measurements and counts are unreliable |
| Long PDF (tender, contract, manual) | Summarise, list requirements, answer questions | Tables and charts may be misread; check key figures |
| Voice notes | Turn a technician's voice note into a job update | Accents, background noise, mixed languages |
| Product photos | Write product descriptions and alt text | Colours and materials can be guessed wrongly |
| Screenshots | Explain an error message or extract a table | Small text; crop to the relevant part |
Our guides on AI product photos for online stores and AI transcription of Singlish meetings go deeper on two of these.
File limits from the vendors' documentation
Limits change often and differ by model, so check the current docs before building anything. As of 21 October 2026, the documentation says:
| Vendor | Images | PDFs and documents | Audio |
|---|---|---|---|
| OpenAI | Up to 1,500 images and 512 MB total payload per request | Each file under 50 MB; 50 MB combined per request | Covered in its separate audio docs |
| Anthropic (Claude API) | Up to 100 or 600 images per request depending on model; 10 MB each; max 8000x8000 px | 32 MB per request; up to 600 pages (100 if context under 1M tokens) | Not a direct input |
| Google (Gemini API) | Supported; large pages scaled to max 3072x3072 | Up to 50 MB or 1,000 pages per PDF | Up to 9.5 hours per prompt; inline requests up to 20 MB |
Sources: OpenAI file inputs and images and vision; Anthropic vision and PDF support; Google document processing and audio. Google notes each PDF page counts as 258 tokens, so a 300-page document is not free to process.
Where multimodal AI gets things wrong
The vendors are candid about weak spots. OpenAI's vision guide and Anthropic's vision docs list similar limitations:
- Small, rotated or low-quality text: Anthropic notes mistakes with low-quality, rotated or very small images under 200 pixels; OpenAI says rotated or upside-down text may be misread.
- Non-Latin scripts: OpenAI says performance may not be optimal for text in scripts such as Japanese or Korean, which matters for bilingual Chinese receipts.
- Charts and styles: OpenAI says models may struggle with graphs where line styles or colours vary.
- Counting and positions: Claude's counts of many small objects are approximate, and spatial positions are rough.
- Medical images: both say their models are not for interpreting diagnostic scans such as CT or MRI.
- Fake images: Anthropic says Claude cannot tell whether an image is AI-generated.
Photos of people, documents and the PDPA
PDPC's Advisory Guidelines on Selected Topics treat photographs and video recordings of identifiable individuals as personal data. Sending them to an AI service is a use and possibly a disclosure, so the usual PDPA obligations apply.
Before sending photos or documents to AI
OpenAI's API data controls say API data is not used to train its models unless you opt in, with abuse monitoring logs retained for up to 30 days unless law requires longer. Consumer chat apps may have different settings, so check which product staff are actually using.
Start small: a one-week pilot
Pick one document type
For example supplier invoices, or defect photos from one site.
Collect 30 real samples
Include poor photos, rotated scans and bilingual receipts.
Write the expected output for each
The fields you need, or the points a good site report must contain.
Run them through two models
Compare accuracy, cost and time. Note every wrong figure.
Decide the human check
Which fields always get reviewed, and who signs off before data is used.
Do it yourself or get help?
Trying a model on a few receipts or PDFs in a business AI plan takes minutes. Building a reliable flow, with files moving from WhatsApp or email into the AI and then into your systems with checks, is where most of the work sits.
Our Workflow Automation (S$390 per workflow, up to 5 steps) connects your apps, for example email attachments to an AI extraction step to a spreadsheet, in your own Make, Zapier or Apps Script account. The AI Website Assistant (S$790) answers customer questions from your own website, FAQ and documents. IT Sorted is a Singapore-based team of developers with over 10 years of experience; you pay upfront, send the brief in writing and follow progress on your order page.
Frequently asked questions
It is AI that understands more than one kind of input, such as text, images, document pages and audio, in the same conversation. You can upload a photo, a PDF or a voice note and ask questions about it in plain language. Current GPT, Claude and Gemini models are all multimodal to some degree.
Often, but not perfectly. Clear printed receipts work well; faded thermal paper, handwriting, rotated photos and mixed-language text cause errors. Use AI to pre-fill fields such as supplier, date, total and GST, and have a person confirm them before anything is posted to your accounts.
It depends on the vendor. As of October 2026, Gemini supports PDFs up to 50 MB or 1,000 pages, Claude's API accepts up to 32 MB per request and up to 600 pages on large-context models, and OpenAI's file inputs allow up to 50 MB per request. Large files also cost more tokens.
Photos of identifiable people are personal data under the PDPA, so you need a purpose customers were told about, reasonable protection and appropriate vendor terms. Use business or API plans that don't train on your data, remove details the task doesn't need, and mention AI processing in your privacy notice. This is not legal advice.
Modern speech models handle many accents, and Gemini accepts audio directly, but mixed languages, slang and background noise still cause errors. Test with your own recordings, keep the audio as the record, and have someone check names, numbers and addresses in the transcript.