Industry · LLM fine-tuning, alignment and evaluation
LLM training data for the languages your model answers badly
Translated English instruction data teaches a model to answer Pakistani questions with foreign assumptions. We produce SFT pairs, multi-turn dialogues, preference rankings, safety and red-team data and held-out evaluation sets written natively in Urdu, Roman Urdu, Punjabi, Pashto and Sindhi by calibrated raters — with the agreement statistics to prove it.
Who this is for
For labs and product teams that claim South Asian coverage and want it to be true
Frontier labs, regional LLM builders and enterprises fine-tuning assistants for Pakistan and its diaspora.
- Frontier and open-weight labs — native instruction and preference data to lift low-resource performance, plus evaluation sets that were not in pre-training
- Regional and sovereign LLM projects — Urdu-first models that need culturally grounded SFT and safety alignment at scale
- Enterprise assistants — banks, telcos, e-commerce and government deploying Urdu and Roman Urdu chat and voice assistants
- Safety and alignment teams — red-team prompts, refusal examples and severity-graded harm data in the local cultural context
- Evaluation vendors — human preference judgments and side-by-side evals by native speakers
What breaks
Why translated data and crowd rating fail
Multilingual benchmarks look fine; real users in Lahore do not agree.
Translation carries the wrong world
An English seed set translated into Urdu asks about Thanksgiving and US tax law in perfect Urdu. Native authoring asks about Eid budgeting, CNIC renewal and load-shedding. See what makes SFT data good.
Rubrics do not translate either
RLHF rubrics written for English encode English politeness, formality and directness norms. Urdu raters applying them produce preference data that fights the culture. See RLHF outside English.
Script and register mismatch
Users type Roman Urdu; corpora are Urdu script. Formal Persianised Urdu in the training set makes the assistant sound like a newsreader.
Rater quality is invisible
Crowd platforms give you whoever signs up. We report per-rater agreement, calibration scores and adjudication rates so you can see whether the preference signal is real.
What we supply
SFT, preference, safety and evaluation data
Native authors and raters recruited to a demographic spec, dual review, and agreement statistics in every datasheet.
Instruction and SFT data
Single- and multi-turn pairs authored natively in Urdu (script and Roman), Punjabi, Pashto and Sindhi, across everyday, finance, health, government, education and technical domains, with refusal and safety examples. See the Native Urdu SFT Corpus.
RLHF preference pairs
Side-by-side rankings with rubric scores, rationales and per-rater agreement metadata; rubrics localised, not translated. See RLHF Preference Pairs.
Safety, toxicity and red-team data
Severity-graded slurs, sectarian and coded language, harassment patterns, and adversarial prompts written by native speakers. See the Safety & Toxicity Lexicon.
Evaluation and parallel data
Held-out native test sets that were never public, side-by-side human evals, and human-translated Urdu–English parallel text in six domains. See the parallel corpus.
Ready-made
Catalog datasets for LLM teams
Each ships with a datasheet, per-record consent artefacts and a free unwatermarked sample pack.
Native Urdu Instruction & SFT Corpus
Human-authored, multi-turn
TextRLHF Preference Pairs
Urdu, Punjabi, Sindhi, Pashto
TextSafety & Toxicity Lexicon
Severity-graded by 3 native raters
TextUrdu–English Parallel Corpus
Human-translated, six domains
SpeechConversational Urdu Speech
For speech-native LLMs
AnnotationModel evaluation
Side-by-side evals and red-teaming on your model
How a programme runs
From rubric to accepted batch
Rubric localisation and rater calibration happen before a single item is written.
Localise the rubric
We adapt your rubric with native reviewers so politeness, directness and harm categories match the culture, then you approve it.
Calibrate raters
Gold items and agreement thresholds; raters below the floor are removed before production.
Author and rank
Native authoring with dual review; preference rankings with rationales; weekly batches with agreement statistics.
Hold-out evaluation
A sealed native test set for measuring your model before and after, never shipped in training data.
Answers
Questions LLM teams teams ask
Behaviour changes usefully from roughly 10,000 natively authored pairs on a strong multilingual base; quality and domain coverage matter more than count. Our SFT data quality guide sets out what to specify.
Both, to your stated proportions. Most Pakistani users type Roman Urdu, so production assistants need it explicitly, with a normalisation convention because Roman Urdu has no standard spelling.
Every datasheet reports per-rater agreement, calibration against gold items and adjudication rates. You can request the raw agreement matrix. See our guide to inter-annotator agreement.
Yes — side-by-side human evals, red-teaming and rubric-scored judgments by native speakers, remotely or inside your perimeter.
Yes — most of our licensing goes to AI teams outside Pakistan. English-language MSA and DPA, USD invoicing by default, W-8BEN-E for US payers, delivery to your AWS, GCP or Azure region. See Where we deliver for compliance mapping by region.
Which language does your model answer badly?
Send a sample of failures. We come back with a data design, native-authored samples and the agreement numbers.
Average first response: under 6 business hours. NDAs signed same day.