For developers and security engineers, the most frustrating aspect of secret scanning isn’t missing a leak; it’s drowning in noise. When an alert system generates too many false positives, trust evaporates. Engineers stop believing the tool, spending precious time triaging obvious mistakes instead of hunting for actual vulnerabilities. At scale, this friction slows down remediation and leaves credentials exposed longer than necessary.
GitHub has addressed this by collaborating with Microsoft Security & AI’s Agents Offense team to inject deeper contextual reasoning into their secret scanning verification process. By moving beyond simple pattern matching to understand how potential secrets are actually used, they have significantly reduced low-value alerts without sacrificing coverage.
Current capabilities and the new challenge
Today, GitHub’s secret scanning relies on two pillars: pattern-based detection for known formats like API keys, and AI-powered detection for unstructured secrets like passwords. This hybrid approach protects tens of millions of developers across billions of code pushes.
While pattern matching is already highly precise, the challenge with AI-based detection was maintaining that same high standard. The goal was to ensure that alerts generated by generic AI detection were just as trustworthy as those confirmed by strict provider patterns.
Building trust through reasoning
The core philosophy is simple: an alert is only useful if you can quickly decide whether it requires action. To achieve this, GitHub enhanced the verification step with contextual reasoning. Instead of just flagging a value that looks like a secret, the system now evaluates how that value appears in the code.
“By looking at how a detected value appears in code, the system can better separate real exposures from values that only look sensitive.”
This distinction is vital. A string resembling a token might be a random UUID, a test placeholder, or an opaque string variable. Without context, these trigger false alarms. With context, the system can filter them out.
Smarter context, not more data
Intuitively, one might assume that improving accuracy requires feeding the AI model larger chunks of code or entire repositories. However, this approach introduces too much noise and increases latency and cost.
Instead, the team extracted a focused set of high-signal information. The system identifies specific usage patterns: is the value assigned to a variable and subsequently passed into an API request, an authentication header, or a database client call?
This method allows the model to distinguish real exposures from false alarms without reviewing the entire file or repository. It focuses on execution paths and usage signals rather than raw data volume.
Results: a 75.76% reduction in noise
The team evaluated this approach against hundreds of customer-confirmed false positive alerts. The target was a 65% reduction in noise.
The outcome exceeded expectations: false positives dropped by 75.76%. This improvement translates directly to the developer experience. With fewer irrelevant alerts, engineers can trust the tool more, spend less time investigating noise, and prioritise fixing genuine security risks faster.
What comes next
GitHub is now expanding this approach to larger datasets and live traffic, refining how context is extracted and utilised for verification. The focus remains on improving signal quality where it matters most: making alerts easier to trust and act upon.
Key takeaways
- Reducing false positives in secret scanning requires better contextual reasoning, not just analysing more code.
- By focusing on usage signals—such as API calls and authentication headers—the system filters out random strings and test data effectively.
- The new approach achieved a 75.76% reduction in false positives, surpassing the initial 65% target while maintaining strong detection coverage.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
