Robert Carr’s story is of the sort that is often branded uniquely American. With roots as a software writer turned entrepreneur, Carr nurtured the company he co-founded from nascent idea into rising payment systems juggernaut. Then, almost overnight, he nearly lost it all.
You see, Robert Carr is the chief executive and co-founder of Heartland Payment Systems, a company that requires little introduction within the IT security circle. Although on its face the Heartland story may not be viewed as a success worthy of profiling, a lesson in perseverance became apparent after sitting down to speak with the company’s CEO at its Princeton, New Jersey, headquarters.
Robert Carr is the sort of person who is well trained to give short answers, likely the result of advice he received from legal counsel following his company’s near-disastrous flirtation with capitalism’s abyss. As I learned from spending a bit of time with the company’s CEO, Heartland’s future was definitely in doubt following its role in the data breach incident that, to this date, ranks as the world’s most severe.
Living a Nightmare
Carr says Heartland first officially confirmed the breach that befell the payment processor’s systems on January 16, 2009, the Friday before a long three-day weekend. This came after being alerted in October of the previous year to the suspected intrusion by both Visa and MasterCard.
“We tried to find evidence of it for a couple of months”, recalls Carr, who, in characterizing the event, says that professional criminals targeted Heartland’s corporate systems because of the large volume of transactions his company processes. “They were able to get into our corporate system and eventually, after about six months of hard work, were able to figure out how to get over into our payment system.”
| "I would always say that the worst thing that could ever happen to us would be to be breached" |
I asked Carr what his initial reaction was to this evidence. The first question he asked himself was would the company survive?
The Heartland CEO then confides that, upon taking his company public, there was only one thing that kept him awake at night.
“I would always say that the worst thing that could ever happen to us would be to be breached” Carr laments, almost as if to sigh a little ‘I told you so’. And breached Heartland was, to the tune of more than 100 million credit and debit card numbers.
He admits that the breach was exactly the sort of thing he knew could happen. After the breach that affected Hannaford Bros. in 2008, Carr told his board that they would likely need to put in place a more robust security strategy.
“We need to have a better security system in this industry”, he recalls telling the board. “If a sniffer can be put on a line in a grocery store’s checkout counter, and intercept card numbers, [then] there are major vulnerabilities in the existing system”.
“I didn’t really know it could happen until it happened at Hannafords”, Carr divulges. “I didn’t realize that the technology was that sophisticated, that they [hackers] could get in and worm their way through, get to the point-of-sale equipment, get around all the anti-virus and all that. I was sort of dumbfounded at what happed to Hannaford, because the other breaches that I had learned of to that point were basically of store data.”
He immediately recognized how serious the problem was, and began giving speeches on the topic prior to his own brush with a data breach. What became apparent quite quickly to Carr was that “you shouldn’t store data in the clear”, and that the likely solution was encryption at the point of sale.
It appeared to me that one of Carr’s biggest regrets over the whole incident was not being able to come up with a suitable solution before hackers compromised Heartland’s systems. Prior to the breach, the CEO spent more than eight months seeking an encryption solution, but came up short on the business terms.
A Call to Arms
Having created the original processing software that was the basis of Heartland, and ushering the company through to becoming publicly traded, it is obvious that the data breach incident affected Carr personally. After all, Heartland Payment Systems was really his baby.
The day Heartland announced the breach its share price stood at $14.18 upon close. By March 6, less than two months later, the price slid to $4.03 per share.
“I was a very large stockholder in the company, and being an entrepreneurial, risk-taking guy, I had borrowed $3 a share against my stock, which was, at the time, less than 10% of its value”, Carr says. “When our stock got below $3.50 cents, I had to pay all my loans back, and I had to sell all my stock. So I went from being the largest shareholder in the company to now a non-shareholder in the company. So it was pretty devastating.”
Regardless, Carr claims this was hardly what concerned him the most about the whole ordeal. He remembers that upon learning of the breach, his team went about creating a plan of action to save the company’s customers and employees, “and then fight the battles necessary to get through”.
“You do your best, and if you don’t win, what’s the worst thing that can happen?”, he asks rhetorically. “You don’t have any money, and you’re broke. Well, I grew up that way, so I’m not terrified of that.”
Next Carr assures me that being flat broke was not his idea of an optimal outcome, but that his real concern was for the more than 3500 employees of Heartland, and what appeared to be their uncertain future.
Prior to announcing the breach publicly, Carr and his team began preparing a rather frank message to keep all of his employees informed. “[We] had a call to action for our employees, because their jobs were all impacted, and I had to honestly say I didn’t know if we’d survive. I was saying that for the next four months, even longer I think. It wasn’t clear at all up until the summer that we were going to survive.”
Survive Heartland did, although not unscathed. More than a year and a half after the breach announcement, Carr tells me that his company has certainly lost a few customers, but that the vast majority of them have remained.
He adds that what Heartland really lost in the aftermath of the data breach was its energy as a company. “We went from a company that was growing very rapidly, organically, because we had a superior business model, to a company that was playing defense just to survive, and we lost some momentum”, Carr admits. “But now that we’ve paid all the card brands, and it’s behind us, we’re regaining that now”.
But Carr seems satisfied with the ultimate outcome after this period of turbulence. His company’s reaction to the breach incident allowed it to retain nearly all of its employees. “So that’s the thing I’m most proud of”, he shares. “Nobody lost their job over it, except one or two people”, Carr says with a half-crooked smile.
Upon Further Review
Carr was eager to share his assessment of employee morale at Heartland following the breach incident. He believes the situation energized everyone at the company, as many employees joined in the fight to save the business.
“We all think we’ve gone through the wars together, and I think we’re ready to fight the next battle, but it will be an offensive battle to capture more market share, instead of playing defense in order to survive. It’s more fun to play offence”, he says, with a rather sly grin.
But now the conversation turns toward post-incident assessment, and Carr shares what he and his company have learned from the hacking incident that lead to the massive data breach.
“Well, what happened to us could have happened to virtually anybody”, Carr asserts. “If an organized criminal group wants to go after somebody, they’re going to get them most of the time, and that’s what happened to us.”
| "I went from being the largest shareholder in the company to now a non-shareholder in the company. So it was pretty devastating" |
What he tells me next is that, regardless of his opinions on the nature of cybercriminals, Heartland has taken tangible steps to prevent a similar incident from occurring again.
“We’ve completely separated our corporate network from our payments network, which was not required by the [PCI] rules. We were trying to live by the rules, we did live by the rules, we think. We’ve put in a lot more monitoring and a lot more data loss prevention – it’s technology – into our system. We hired a lot more people to watch things on a regular basis.”
Carr has been an outspoken critic of the QSA system, which is understandable since Heartland was certified as PCI compliant prior to the breach. Although he trusts that the QSA system is making positive strides, he still believes the system is not all it’s cracked up to be.
“I think that the PCI Council has taken significant action to upgrade the quality of the QSAs, but I think any company that relies on their QSA to say that they’re secure is operating with false assumptions, but we’re not supposed to tell them. That’s better than saying their reports aren’t worth the paper they’re written on.”
Carr says that QSAs are good at flushing out the obvious neglect, but that companies processing card transactions should by no means feel completely comfortable simply because they passed a QSA audit.
Likewise, he sees value in PCI compliance, but calls it a framework for what should be done at a minimum, and that companies processing transactions need to take additional steps to secure data. The optimal solution, in Carr’s opinion, is end-to-end encryption, which begins at the card swipe and protects data right through to the gateway.
He also sees tokenization as a good solution, but still prefers end-to-end encryption because tokenization does not, in his view, protect transactions at their most vulnerable point – at the merchant.
It is fair to say that Carr’s experience in dealing with a data breach provides him with a unique perspective on the topic, and he concludes the interview with words of wisdom for organizations that may find themselves in a similar predicament. He recommends two things, without hesitation: find an experienced law firm – one with the requisite experience in similar cases – and bring in a security advisor that really knows their stuff.
He jokingly says he must have received thousands of calls from vendors and other people in the security business, all claiming to be ‘experts’. Carr says that a company that believes it’s the victim of a breach really needs to bring in a qualified security advisor to determine the cause and then mitigate those causes.
Collecting all the data that’s possible concerning the incident is another key, because, as Carr warns, you need to put yourself in the best possible position to make an appropriate legal defense if you wind up in court.
It’s all sound advice for sure, from a general with the requisite combat experience.