AI Voice Calling Bot (Real-Time Lead & Worker Interaction)
Streaming AI voice bot for automated worker assessments using WebSockets and OpenAI Realtime API.
Overview
A real-time AI voice calling system built to automate the worker onboarding assessment process at Broomees. The bot conducts live phone-like conversations, evaluates responses, and scores workers automatically.
Architecture
- Streaming Voice Pipeline: Audio streamed via WebSockets with sub-200ms latency
- OpenAI Realtime API: Used for speech-to-text and response generation
- Assessment Engine: Custom scoring logic based on conversation analysis
- Integration: Plugged into the worker onboarding flow seamlessly
Technical Highlights
// WebSocket audio streaming pipeline
const audioStream = new WebRTCStream({
sampleRate: 24000,
channels: 1,
encoding: 'pcm16'
});
The system reduced assessment time by 70% and enabled 24/7 automated onboarding.
Results & Metrics
Key Features
AI-Powered Search
Semantic search using embeddings for smarter results.
Cross-Platform
Native feel on iOS, Android, and the web.
End-to-End Encryption
All user data encrypted at rest and in transit.
Real-Time Sync
Live updates via WebSockets across all devices.
Development Process
Discovery & Planning
Defined scope, mapped user flows, and set up the monorepo.
1 weekDesign System
Built tokens, components, and Figma prototypes.
2 weeksCore Development
Implemented authentication, data layer, and main features.
5 weeksTesting & Launch
E2E tests, performance tuning, and staged rollout.
1 weekChallenges & Solutions
My Role
Lead Engineer & Designer
I was solely responsible for the full product, from initial concept to production deployment.
- →Architected the backend API and database schema
- →Built the design system from scratch in Figma
- →Implemented CI/CD pipeline with GitHub Actions
- →Conducted user interviews and iterated on feedback
Related Projects
Tech Stack
Links
Private repository