Nelvis: Whisper AI Model Transcription Automation and ASR Dataset Creation

Overview

Nelvis Assurances & Crédit needed a Python / AI developer to automate transcription and build ASR (Automatic Speech Recognition) datasets — the labelled audio-and-text pairs used to train and evaluate machine-learning models that convert spoken language into written text. The audio came from real insurance conversations: inconsistent formats, variable recording quality, French domain vocabulary and no reliable transcripts. I built an end-to-end pipeline in Python running the Whisper AI model on RunPod GPU instances: ingestion from cloud storage, GPU batch transcription, translation and repair passes, quality checks, and export of structured dataset files ready for model training and evaluation.

The challenge

• Thousands of raw audio files with mixed formats, sample rates and recording quality, and no ground-truth transcripts to start from.
• Audio files were in multiple African Languages — Lingala, Swahili, Kituba, Sango, French.

What I built

• An ingestion worker that pulls audio from cloud object storage and normalises every file to a single processing format.
• A GPU transcription worker running the Whisper AI model on RunPod, batching jobs so large backlogs finish in hours rather than weeks.
• Dedicated translation and repair workers that re-process weak or failed segments instead of rerunning the whole batch.
• A backfill and statistics worker that tracks coverage, flags gaps and reports dataset quality metrics per batch.
• Export workers producing structured ASR dataset files — aligned audio and text pairs with metadata, ready for training and evaluation splits.
• Dockerised workers with environment-based configuration so the same code runs locally and on GPU instances unchanged.

Results

• Transcription moved from a manual, one-file-at-a-time task to a fully automated batch pipeline.
• GPU processing on RunPod cut turnaround from weeks of CPU time to hours per batch.
• Produced reusable ASR datasets that can train and benchmark speech-to-text models on the client's own domain audio.
• Failure recovery is targeted: only the affected files are reprocessed, keeping GPU spend proportional to real work.

Tech stack