Speaking at Infosecurity North America on “Demystifying the Science of Machine Learning in Endpoint Security” – Jack Danahy, CTO of Barkly pointed to examples of how machine learning was used in sustaining Bordeaux wine and in baseball (as dramatized in the movie Moneyball), and said that as it is “used all over the place”, he said that often people ask what it means, what does it do and how can users take advantage of it.
Danahy said that having data and knowing what to look for has become “so popular as computers got so powerful and are churning through tons of questions on data”, and we either need computers that are powerful and have enough resources to gather enough data to make sense, or a better solution.
“The data also has to be good but there is also a limit on what you’re looking for: in the case of Moneyball [Billy Bean] was not sure what he was looking for but with security, think about looking at data, and the first challenge is what to look for within the data and what feature or factor will we look for.”
Danahy said that machine learning starts with supervision, which means teaching the system on one set of samples whether they are good or bad, and samples are arranged into training sets and the job of machine learning is to learn from those samples. “With tens of thousands of samples, how to train those to categorize them in right way,” he said. “It allows it to distinguish and learn how to judge about being in one camp or another.”
This is followed by unbalanced training, and unsupervised training where clustering comes in.
In terms of the endpoint, Danahy said that attackers get on to systems and with new attacks, attackers do their very best to find ways to obscure that they are running and dwell time is a huge problem. “With new forms of attack like ransomware, it takes less than 20 seconds [to infect], so that is why machine learning makes sense,” he said. “It can be really fast if I am smart on the way I train and give it the information it needs and background on the information I need.”
Danahy said that security creates a lot of messages, but models don’t live forever—so think how much malware changes with 365 million new samples a year.
He said: “There will be a new swing of stuff that your machine is not trained on, so the model you are using begins to deteriorate and has a ‘half life’, and with old malware out there you can train on what no one uses but that is perhaps not the best model.
“The model is that every day, everything is new, and retrain model so applied for protection to minimize the threats, and deliver accurate protection.”
He concluded by saying that there is new technology to gather samples and monitor transactions, and this is probably the only way to keep up with malware and probably a way forward for the consumer, “but also a way for security to leapfrog the bad guys”.