Katie Moussouris is no stranger to the world of bug bounties having helped Microsoft create its first program and now as leader of her own company with Luta Security.
In a session at the Black Hat USA 2022 security conference, Moussouris detailed the history, challenges and potential future of bug bounties.
The first modern bug bounties really got started in 1995 with Mozilla offering rewards for security flaws. In 2013, Moussouris helped to create Microsoft's bug bounty program, which at the time was the highest offered by any industry vendor in the world.
"I like bug bounties I cannot lie, but the fact is they have not delivered on their great promise," Moussouris said. "We wanted them to have revolutionary security benefits, keep hackers out of jail and get them paid and build the cyber workforce pipeline of tomorrow."
There are a number of reasons why bug bounties haven't lived up to Moussouris' expectations. The primary one is a lack of organizational commitment to actually fix bugs.
"You would not believe how many organizations we see that are performing what I call bug bounty Botox," she said. "They launch a bug bounty, and they check a box saying they have a program and they literally use the platform terms of non disclosure to lock all those bugs away – they might pay for them, but they're not fixing them."
Key Metrics to Measure Bug Bounty Success
Moussouris identified several key metrics that can be useful to help organizations determine if a bug bounty program is effective for them.
The first metric she identified is Meant Time to Repair (MTTR), which analyzes how long it takes organizations to fix bugs of different severity. This information is also valuable to understand the volume of duplicate bug entries that come in, which could indicate how discoverable a vulnerability might be.
Another metric that is useful is to track is the case reopen rate. This determines how efficient the program is at actually properly resolving issues.
"Understand that fixing bugs themselves is treating the symptoms of your underlying security problems," she said. "Fixing your processes is the cure and anticipating the places where your processes are going to need help."