Heartland Sniffer Hid In Unallocated Portion Of Disk

The sniffer malware that surreptitiously siphoned tons of payment card data from card processor Heartland Payment Systems hid in an unallocated portion of a server's disk. The malware, which was ultimately detected courtesy of a trail of temp files, was hidden so well that it eluded two different teams of forensic investigators brought in to find it after fraud alerts went off at both Visa and MasterCard, according to Heartland CFO Robert Baldwin.

"A significant portion of the sophistication of the attack was in the cloaking," Baldwin said.

Payment security experts pretty much agreed that hiding files in unallocated disk space is a fairly well-known tactic. But it requires such a high level of access—as well as the skill to manipulate the operating system—that is also indicates a very sophisticated attack. One of those security experts—who works for a very large U.S. retail chain and asked to have her name withheld—speculated that the complex nature of the hiding place, coupled with the relatively careless leaving of temp files, could suggest a less-skilled cyberthief who simply obtained some very powerful tools.

But she cautioned against reading too much into whatever clues the culprits left behind, given that some might be deliberately misleading. "Anyone who has access to that level of the machine can make it look like anything they want," said the retail security manager. "There is virtually no way to tell in a case like that what really happened. If they have a chance to lay down false trails, it's pretty hard to find out what really happened."

Consultants agreed that this type of attack would require extensive access and the ability to trick the machine into believing the thief has very significant user privileges. But it wouldn't necessarily require modification of the OS directly. "They could have done it two ways. You can modify the OS or you can install a modified device driver."

Another consultant—who also wanted his name left out—said the ability to write directly to specific disk sectors is frightening. "Somehow, these guys went directly to the base level of the machine (to an area) that was not part of the file table for the disk," he said. "Somehow, they got around the operating system. That's a scary mother in and of itself."

More Details Emerge

Baldwin also added more details to the sketchy timeframes that have been revealed thus far about the attacks, specifying that Heartland was contacted by Visa and MasterCard "in very late October," possibly October 28. The card brands had been unable to find a common point of purchase with a series of bogus cards, so they started to look for a common point of processor, which took them to Heartland.

The processor was then given a pile of problem cards to try and match. Baldwin said he didn't know how many account numbers they were given, but said "it was at least in the hundreds. Not sure if it was more than a thousand." Heartland quickly discovered that some of those names hadn't even used any of Heartland's networks, although many had.

Some of the problem might not involve common points of purchases or common points of processors as much as common points of high-tech hoodlums. Baldwin said Justice Department and U.S. Secret Service officials have told him "the bad guys they think got us have successfully breached other financial institutions."

Apparently, federal law enforcement was focusing on suspects in other breaches when the Heartland breach became known, which explains the relative speed of the Secret Service identifying a key suspect, apparently in Eastern Europe.

And Heartland's civil legal troubles are just starting, with one of the least surprising lawsuits ever filed. The litigation is the start of a class-action lawsuit that accuses Heartland of having "failed to take appropriate measures to adequately protect" its data. This new lawsuit, filed Tuesday (Jan. 27) on behalf of a consumer named Alicia Cooper, will likely run into the same issues that blocked the TJX class-action lawsuits. With zero liability cards, it's unlikely the plaintiffs will be able to prove monetary damages. In today's civil courts, that's the ballgame.

Back to the timeline. After the card brands alerted Heartland in late October, it took about two weeks of internal investigation to conclude that, yes, the company had been breached. The initial internal conclusion was that "it looked most likely that it would be in a certain segment of our processing platform," said Baldwin, adding that Heartland does not want to identify what that segment was. The company hired a forensic investigation team to come in and focus solely on that one area, an effort that ultimately proved fruitless. "We found issues in a large segment of our processing environment. The one that looked like the most promising turned out to be clean," he said.

While the first team was working, Heartland had a second forensic team brought in to check the entire system. "That first firm had a very specific scoping of their assignment. The second firm was working in parallel on the rest of that processing."

That second team "was nearing conclusion" and was about to make the same assessment the first team did: clean bill of health. But one of the last things that external, qualified risk assessor did was to try and match various temp files with their associated application. When some orphans—.tmp files that couldn't be matched to any application or the OS—were turned over to Heartland's internal IT group, they also couldn't explain them, saying that it was "not in a format we use," Baldwin said. More investigation ultimately concluded that those temp files were the byproduct of malware, and more searching eventually located the files in the unallocated portions of server disk drives.

Heartland officials said it won't be known for certain who was behind this attack until all the investigations are complete. However, preliminary indications are pushing them to suspect a fully external attack, with no indications at this time of any help from any Heartland employee or contractor. "The existence of a key logger, that could certainly have been by an outsider," Baldwin said.

Developing End-To-End Encryption

Heartland on Tuesday (Jan. 27) announced that it will be creating a new department that will be "dedicated exclusively to the development of end-to-end encryption."

"PCI is a good and effective standard, but the bad guys have become more sophisticated to the point where encryption of data in motion appears to be one of the next required steps. There is no single silver bullet that will secure payment systems, and constant vigilance and monitoring of the infrastructure will always be required," Heartland CEO Robert Carr said in a statement. "Nevertheless, I believe the development and deployment of end-to-end encryption will provide us the ability to implement increasing levels of security protection as they become needed."

End-to-end encryption is far from a new approach. But the flaw in today's payment networks is that the card brands insist on dealing with card data in an unencrypted state, forcing transmission to be done over secure connections rather than the lower-cost Internet. This approach avoids forcing the card brands to have to decrypt the data when it arrives.

Given that, most so-called end-to-end encryption approaches leave a small window of opportunity before the card is encrypted, and right after it's decoded. The goal for most companies is to simply shrink that insecure window to as short a time period as possible.

CFO Baldwin was asked whether a more airtight resolution would be preferable. "The more players you have to get to change their behavior, you grow the challenge to get any change implemented exponentially," he said, adding that "the actual amount of losses due to fraud is at a very, very low level," which forces "an appropriate cost-benefit analysis. For the system as a whole, it just may not be worth it (to try and do a complete overhaul). We're reducing that window rather dramatically, working with a limited number of players."

Baldwin also said that a more aggressive effort would "slow down some systems in very significant ways. It would be noticeable."

Clarification From Heartland

After this article was published, Heartland sent us a note and a call to ask us to clarify some details.

The story above reports that, ''after the card brands alerted Heartland in late October, it took about two weeks of internal investigation to conclude that, yes, the company had been breached." Heartland spokesperson Jason Maloni said that Heartland didn't conclude that they had been breached at the end of the two weeks, but had concluded that they might have been breached and chose to bring in two forensic teams. The distinction is whether, at the end of the two weeks before they brought in the two forensic teams, Heartland believed that it had been breached or merely suspected that it had been breached.

The story above also reports that the malware "eluded" those forensic teams and that, at the last minute, a cleanup of temp files discovered the rogue program's trail. Maloni says that given that the trail was ultimately discovered by one of those teams, a better phrasing would have been that the program had temporarily eluded the teams.

The most significant—and vague—clarification request involves the specifics of where the malware was discovered. Maloni now says that "the forensic report with these answers is not yet complete" and that "CFO Bob Baldwin misspoke on this point." Despite Baldwin's comments in an interview that the malware "hid in an unallocated portion of a server's disk," Heartland is now saying only that some parts of the program were in the unallocated portion of the server's disk, but that "substantive elements of the malicious software" were also located elsewhere. Maloni didn't say what parts were where not what percents nor what he considered "substantive elements."