Live Human Detector on Outbound Phone Calls [R]

“`html

Introduction

The goal of this project is to save humans from wasting time in call center queues by implementing a tool that can classify audio within a sub 1-2 second window, determining whether calls have transitioned out of the queue and into a live person.

Approach

To achieve this, we will use machine learning to analyze the acoustics or spectrogram (via Fast Fourier Transform) of an audio stream. The tool must be able to classify audio in real-time with high confidence levels. This approach does not rely on traditional speech-to-text (STT), as additional layers for labels such as Voice-Recorded Announcement (RVA), Text-To-Speech (TTS), Voicemail, and call screening will be added later.

Phases

• Queuing: Labels include Music, TTS, RVA (Recorded Voice Announcement).
• Transitioning: Labels are Ringback, Answered, and Machine Beep.
• Connected: Human speech, Fax, Voicemail, Call Screening.
• Disconnected: Engaged Tone.

References

• YOHO: You only here once
• Vicidial forum discussion on call center analytics
• Hugging Face audio classification pipeline tutorial
• Google AI Edge MediaPipe samples for audio classifier
• Scikit-Learn machine learning map
• Research paper on audio classification techniques

Questions and Next Steps

• What is the best framework / algorithm to start with? Existing frameworks like YamNet have shown good performance for real-time audio classification. Other options such as Whisper (for STT) or ASR might be useful but are not recommended at this stage.
• How should I label and structure my data? Existing full-length recordings can be labeled with stop/start timestamps, but splitting each label into its own file could result in a loss of context. We need to decide based on the specific requirements and constraints of our project.
• Are there existing datasets available for training? While we don’t have specific datasets provided here, it is crucial to find or create appropriate labeled audio data that matches the criteria of different call states (Queuing, Transitioning, Connected, Disconnected).

Key Takeaways

• The tool must classify audio within a sub 1-2 second window for real-time decision-making.
• We will use spectrogram analysis to identify and label different call states accurately.
• Existing datasets like those from the Vicidial forum or research papers on audio classification can be leveraged for training our model.

“`

Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.

Please enable JavaScript in your browser to complete this form.

Name *

First

Last

Email Name

Email *

Submit