ISP Dynamic IP Address Switch Knocks Dollar Tree Store Off The Network For Two Days

A mistake by an Internet service provider recently disconnected a South Carolina store within the $4.6 billion, 3,803-store Dollar Tree chain from its corporate network for two days, an unusually long duration for such an outage. Making matters even worse, some store employees interpreted the outage as a cyberattack and shared that with both customers and the media.

The chain, which has stores in 48 states in the U.S., lost its data connectivity on December 10 when the service provider made a modification that went awry. "It was an inadvertent switch to dynamic IP from fixed IP," said CIO Ray Hamilton. "That shouldn’t happen, and it took us a couple of days to resolve."

The CIO said the store continued to function, with payment card information handled "in a proprietary manner," although E-mail and all other information normally shared through the corporate network were unavailable. Payment card transactions, inventory and other data were handled locally until the network came back up on Saturday (Dec. 12).

After employees talked with a local television station in North Augusta., S.C., a report was aired that the store's IP address was "stolen" and that customer information had been exposed. (Like we don't see enough of those reports when it really happens?)

Dollar Tree Vice President of Investor Relations Tim Reid said the service provider's goof meant that the main Dollar Tree servers could not recognize the store's system because of the IP address change. Hamilton declined to mention the name of that provider. "We are in all 48 states," he said. "We use a little bit of everybody."

He said the North Augusta store disconnection wasn't initially a priority for the Dollar Tree IT department. "It took a couple days," he acknowledged. "With 4,000 stores, I probably have one or two that don’t connect every day. When they get to two days [of being disconnected], everybody is looking at it."

Reid said no customer personal data was lost during the outage. "The bottom line is, what happened was the communication link between the store and the network was down temporarily," he said. "It was a loss in the store's data connection. It went down Thursday, and it got resolved on Saturday. All data was secure at all times and remained so. Somewhere along the line, employees spoke directly to this reporter or somebody else, and the facts got garbled."