Punjabi · Shahmukhi · ~90 million speakers in Pakistan
Punjabi AI training data: speech, text and annotation
Punjabi is one of the ten most spoken languages on earth and one of the least represented in AI. We collect it at source in Punjab — spontaneous speech, telephony audio, studio TTS, native-authored Shahmukhi text and rater judgments — consented per record and licensed for model training.
At a glance
Punjabi training data: the facts a buyer needs first
The numbers that decide whether a Punjabi programme is feasible, before anyone writes a statement of work.
| Speakers | ~90 million in Pakistan (largest language in the country); ~30 million more in India; large diaspora in the UK, Canada, USA and Australia |
|---|---|
| Script | Shahmukhi (Perso-Arabic, written in Nastaliq) in Pakistan; Gurmukhi in India. Speech is shared across the border, script is not |
| Varieties we field | Majhi (Lahore, standard), Shahpuri, Pothwari (Rawalpindi – Pothohar), Jhangvi and Dhani on request |
| Where we collect | Lahore, Faisalabad, Gujranwala, Sialkot, Sheikhupura, Rawalpindi and rural districts across central and northern Punjab |
| Capability tier | Deep speech · Deep text · Standard image/video |
| Typical pilot | 50–200 hours of speech, or 25k–100k text items, in 3–5 weeks; creditable against production |
| Licence | Perpetual, worldwide, sublicensable; derivative-model rights stated explicitly. Licensing detail |
| Delivery | Your S3, GCS or Azure bucket with manifests, checksums, QA report and datasheet |
Speaker figures are approximate, drawn from the 2023 census and public sources, and include second-language speakers where noted.
What you can license today
- Conversational Punjabi speech with verbatim transcripts and diarization
- 8 kHz Punjabi telephony audio from the telephony corpus
- Punjabi wake-word and command utterances
- Native Punjabi RLHF preference pairs and toxicity lexicon
- Custom: TTS voices, Shahmukhi OCR, Punjabi–English parallel text
Built for
- Fine-tuning Whisper, MMS or wav2vec 2.0 for Punjabi ASR
- IVR and voice agents for Pakistani, UK and Canadian Punjabi speakers
- LLM fine-tuning and evaluation in Shahmukhi Punjabi
- Content moderation and safety classifiers for Punjabi text
Why models fail
Why Punjabi breaks models trained on Urdu or Hindi
Punjabi is often treated as a dialect of Urdu or Hindi in data budgets. Acoustically and lexically it is not, and the shortcuts show up as word error rate.
Tone
Punjabi is tonal — roughly three lexical tones distinguish otherwise identical words. Urdu and Hindi are not. A model transferred from either flattens the distinction and confuses common word pairs.
Almost no written corpus
Punjabi speakers in Pakistan overwhelmingly write in Urdu. Shahmukhi web text is a fraction of a percent of Urdu text, and much of it is poetry or religious material, so text models see none of the everyday register speech models need.
Two scripts, one speech
Shahmukhi and Gurmukhi Punjabi share speech but not orthography, so Indian Punjabi text data does nothing for Shahmukhi tasks. Speech collected in Pakistan does transfer to Indian Punjabi ASR — one of the few places where Pakistani data covers two markets.
Heavy Urdu and English mixing
Urban Punjabi speech switches into Urdu and English mid-sentence. Transcription conventions have to state how each language and the switch points are written, or two transcribers will produce two different ground truths.
What we collect
Punjabi speech, text, image and annotation data
Every stream is quota-controlled: we recruit to a demographic and acoustic spec you approve, in the districts where the speakers actually live, and reject out-of-quota submissions even when they are technically clean.
Speech for ASR
Spontaneous conversation, telephony, scripted read speech and command utterances. Quotas across age, gender, urban/rural, variety and device. Verbatim transcripts with a written code-switching convention, speaker diarization and noise tags.
Speech for TTS
Single- and multi-speaker studio recordings in Majhi Punjabi, phonetically balanced scripts, 44.1 kHz, with pronunciation lexicon. 10–40 hours per voice; see how much a TTS voice needs.
Text, SFT and RLHF
Native-authored Shahmukhi instruction data, multi-turn dialogues and preference pairs written by Punjabi speakers, not translated from English. Punjabi–English and Punjabi–Urdu parallel text in six domains.
OCR, image and annotation
Shahmukhi printed and handwritten pages for OCR, Punjab street and retail imagery, and human labeling or evaluation on data you already own, by native Punjabi raters.
Ready-made
Catalog datasets that include Punjabi
Each ships with a datasheet, per-record consent artefacts and a free sample pack. Subsets by language are available on request.
Pakistani Telephony Corpus
8 kHz call-center audio in Urdu, Punjabi, Sindhi, Pashto
SpeechWake Word & Command Set
Urdu, Punjabi, Pashto and English commands
TextRLHF Preference Pairs
Native rater rankings in Urdu, Punjabi, Sindhi, Pashto
TextSafety & Toxicity Lexicon
Severity-graded terms in Urdu, Punjabi, Pashto, Sindhi
SpeechPakistani-Accented English
L1-tagged, including Punjabi-L1 speakers
SpeechConversational Urdu Speech
Companion corpus for Urdu–Punjabi bilingual systems
Specification
Writing a Punjabi spec that survives QA
Most Punjabi projects that fail do so in the specification, weeks before the first recording. These are the decisions to make in writing.
State the variety and quota
Majhi alone, or Majhi plus Pothwari and Shahpuri in stated proportions. Tie the quota to districts, not just provinces.
Fix the script policy
Shahmukhi throughout; whether English and Urdu tokens are transliterated or kept in source script; how tone is not marked (it never is in orthography) and how that affects evaluation.
Write the code-switching convention
Tag language spans or not; treat Urdu loanwords as Punjabi or Urdu; how to transcribe hybrid verbs. Two pages of rules, agreed before QA starts.
Define acoustic conditions
Telephony 8 kHz versus wideband; share of far-field, in-vehicle and market-noise audio; minimum device diversity.
Set the acceptance test
Held-out test set drawn from the same quota, target WER against a reference transcript, inter-annotator agreement floor on a 5% double-transcribed sample.
Who buys Punjabi data
Punjabi is a domestic language in Pakistan and a diaspora language in the UK, Canada, the US and Australia
That is why Punjabi data is bought from Lahore and from London. The use cases overlap far more than buyers expect.
- Voice agents and IVR — banks, telecoms and utilities in Pakistan; NHS, council and utility services in the UK, where Punjabi is among the most spoken non-English languages
- Speech-to-text and translation — remote interpreting, healthcare and legal transcription for Punjabi-speaking communities in Canada, the US and Australia
- LLM localisation — Shahmukhi instruction tuning and evaluation for assistants serving Pakistani Punjab
- Content moderation — severity-graded Punjabi toxicity and hate-speech data for platforms with South Asian audiences
- ASR for Indian Punjabi — Pakistani Punjabi speech as the acoustic base for Gurmukhi-output systems
Answers
Questions buyers ask about Punjabi data
Standing panels in Lahore, Faisalabad, Gujranwala, Sialkot and Rawalpindi let us deliver a first batch of 50–200 hours within weeks and scale to 1,000+ hours over a programme. Ready-made Punjabi audio is licensable today inside the Pakistani Telephony Corpus and the Wake Word & Command Set; request a free sample to check format and quality.
Shahmukhi, because that is what Punjabi speakers in Pakistan read and write. Gurmukhi transcripts can be produced by transliteration and native review for buyers targeting Indian Punjabi; the speech itself transfers with no change.
Yes. Spoken Majhi Punjabi is shared across the border with minor lexical differences, so acoustic models trained on Pakistani Punjabi transfer well to Indian Punjabi speech. Text data does not transfer because the scripts differ.
Yes. Pothwari (Rawalpindi–Pothohar), Shahpuri (Sargodha) and Jhangvi are on active or ramp-up panels with a 2–3 week lead time. The capability table on the Languages page states the current tier per variety.
Yes. Most of our licensing goes to teams outside Pakistan. Contracts are in English under a mutually agreed governing law, invoices are in USD by default, and delivery is to your own cloud bucket in the region you choose. See Where we deliver for the compliance and data-residency detail.
Consent is read aloud in Punjabi and recorded, then confirmed in writing where possible. Each record carries a consent version, the contributor pseudonym and a withdrawal route. The full method is on the Consent & Fair Pay page.
Tell us what your Punjabi model is getting wrong
Send a two-line description — the task, the variety, the acoustic conditions. You get a scoped collection design and a realistic timeline back, not a discovery-call funnel.
Average first response: under 6 business hours. NDAs signed same day.