Schneier's main point is that training doesn’t work. “If four-fifths of company employees learn to choose better passwords, or not to click on dodgy links, one-fifth still get it wrong and the bad guys still get in.”
Ira Winkler subsequently took issue, strongly advocating user training. “That argument basically says that if the bad guy gets in, all security countermeasures are irrelevant. By that measure, we should abandon security as a whole, since all countermeasures have and will fail.” But Winkler does accept that there are difficulties: “it is hard to measure the incidents that you prevent. Additionally, few security awareness programs take metrics.”
The inability to measure the effectiveness of training programs is one of their greatest weaknesses. CSOs don’t know where to concentrate their efforts, and have difficulty in providing realistic RoI figures to the purse holders. "Generally speaking," notes a new report from Wombat Security Technologies, "the Security team only learns that training hasn’t worked when it fails – and that’s too late."
But there is one relatively new user training technique that does return metrics: simulated attack training. Wombat's report is based on conversations with and between a number of senior CSOs who have used this technique.
The basic idea behind simulated attack training is that a company should try to phish its own staff. By doing this in a controlled manner, CSOs can discover their weakest links, know where to concentrate additional training, and measure the efficiency of that training over time. "More than anything else," notes the report, "simulated attack training can introduce metrics into training – not only is it effective, its effectiveness can be measured and monitored to allow the most cost-efficient training in the most cost-effective areas."
It gives an example from training implemented by a Fortune 50 company: “Almost 35% of the recipients fell for the first simulated phishing attack but less than 6% fell for the second attack – which demonstrates an 84% decrease in susceptibility.”
"Phishing, and the more targeted and sophisticated spear-phishing, is the weapon of choice for the modern cyber-criminal and is used by the more organized hacker for data and intellectual property theft," comments Perry Carpenter, former security awareness analyst from Gartner who is now working as a security expert in the financial sector. "While there is no foolproof technological defense, contemporary thought now focuses on training the user to recognize and resist targeted social engineering."
The Wombat study provides practical help and advice from senior security executives who have used and are using simulated attack training. And in particular it explains the organizational difficulties in getting management buy-in throughout the business. “Some brave CSOs have even phished their own executives just to prove the point about everyone’s vulnerability,” notes the section on ‘how to implement a simulated attack training program.’
“There is strong evidence that continuous security awareness training that includes simulated attack training works to significantly reduce risk," said Joe Ferrara, President and CEO of Wombat Security Technologies. "As it shows in the report we have seen susceptibility reductions of over 80% when comparing an initial mock attack to subsequent attacks when in-depth training is completed in between the attacks.”