How do you like your facts? Straight up, or alternative? How do you prefer your logic – smooth, or with a twist?
Francis Wheen's "How Mumbo-Jumbo Conquered The World" must rank as one of the least successful polemics of this century. It is unreasonable, though, to expect one person – or one book – to halt society's onward march of unreason.
This is a passionate and joyful defense of scientific rationalism, though science itself is only one aspect of the book.
His first target is management gurus and their works. Of these, first on the list is "In Search of Excellence". Having sold five million copies, it was then flatly contradicted by the same author's "Thriving On Chaos" a few years later (‘there are no excellent companies').
I’ve determined three trends which by then had already started to undermine public belief in science: deliberately unclear communication, the inadequate application of scientific method and seeking blindly for truth. The scientific community didn't cause most of the damage, but has signally failed to halt these trends.
The impact of inadequate science and false facts
Chapter 4 – "The demolition merchants of reality" – sets the stage. Post-modernism is the cue for science to strut its stuff; and, along with it, an early Internet service – the Post-Modernism Generator. After all, if what you write doesn't have to make any sense, why not get a computer to do it? It served more than half a million randomly-generated post-modern essays in the first two years.
This all got rather more serious after Alan Sokal (a physicist) managed to get his hoax "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity" published in Social Text in 1996. I recall that a little later, at the height of the internet boom, there was another service that generated a random start-up company web site – complete with random product descriptions, random mission statement, and random founder biographies. Occasionally it would throw up a random security product that looked uncomfortably similar to the genuine security snake-oil that was around at the time.
The effects of inadequate science can linger well after publication. "The Jupiter Effect" (1974) warned that planetary alignment would cause earthquakes in Los Angeles in 1982. It was a best-seller, and was rapidly discredited by scientists (the effects are far, far, too small).
Even though co-author John Gribbin retracted his position in 1980, the scare was stoked up again as 1982 approached. There was another planetary alignment earthquake scare in 1988 – this time, the Griffith Observatory set up a Nostradamus Hotline to deal with the calls.
The misrepresentation of science and technology
Mathematical prowess proved its worth in another debunking exercise – finding coded prophetic messages in classic texts. (Nostradamus, the Knights Templar, and their ilk had a 1990s revival.) Drosnin's challenge went out: "When my critics find a message about the assassination of a prime minister encrypted in "Moby Dick", I'll believe them."
Brendan McKay of the Australian National University promptly found such a message about Ghandi, using the prescribed equidistant letter spacing technique. In a nice long text, the laws of probability work just fine: seek, and ye shall find.
Which brings us to today's topic: security analytics. Early commentators on security analytics have misrepresented the technology – with the same three tendencies: poor explanations, bad scientific method and the promise of gold. Arm yourself:
Be clear in your aims - The technology is packaged and tuned to meet different needs. Are you trying to detect user impersonation or changes in user behavior? Changes in device behavior? Emerging or latent threats? Insider or outsider threats? Does the solution need to be self-tuning, or do you expect to need some administrator intervention anyway?
Take stock of what you already know - If you use endpoint analysis today, you probably know some of the data that drives security decisions. Any machine learning system will take time to train; meanwhile you can the endpoint analysis data as a baseline
Carry out a scientific trial - Simulate attacks that actually happened on your systems, and see if the security analytics detect it. If so, ask why: if not, ask why not. Are the false positives and false negatives acceptable? Examine carefully any incidents that occurred during the trial period. Determine the predicted savings and reduction in risk that come solely from the security analytics solution.
Understand the limits of the technology - Machine learning systems cannot yet explain their own behavior; however, apart from this black box, your supplier should be able to describe the rest of their offering. You may need this description for regulatory purposes (for example, to meet GDPR requirements).
Mumbo-jumbo is inexcusable in business. In the wider world, Francis Wheen noted at the time that “not everyone wanted to be Donald Trump"; well, up to a point, Lord Copper. "Hilarious" is how Jeremy Paxman described this book. He's not laughing now.