Wal-Mart's Plans For Its 4-Petabyte Database

When the world's largest retailer struggles with a database issue, the numbers can be a bit daunting. It's datawarehouse, for example, is larger than 4 Petabytes. That's more than 4,096 TBytes, give or take a few million bits.

The chain has more than 6,000 stores, with some having almost a half-million SKUs each. You think your Excel spreadsheets are bad? Wal-Mart's database tables have literally 100 billion rows. The retailer's POS systems have to ring up some 276 million items?a day.

This peek into the IT operations of this $345 billion retail empire comes courtesy of a phone interview with Wal-Mart Chief Technology Officer Nancy Stewart. The occasion: Wal-Mart wanted to talk up a recent purchase of HP Neoview systems.

Although HP and Wal-Mart issued the statement on Aug. 1, the systems have been in place for months, having gone into full production in May, Stewart said. Wal-Mart rarely agrees to news releases for purchases, let alone interviews. Why this special treatment for HP?

"We delayed this announcement for several months. This announcement has not come on a whim," she said. The HP system "has very strong high-availability characteristics. The thing that did it for me personally in terms of the relationship is that it is an extremely high availability system and it requites very little support."

HP's servers, which will take their place along with hardware from IBM, EMC and Teradata, were chosen because of how they performed within Wal-Mart's very high-volume, high stress environment."We wanted our relationship to see if we could max out high performance characteristics," she said. "What a lot of the folks we have dealt with do is they will parse the queries. Here at Wal-Mart, we don't batch the queries. We process those records in realtime."

The Wal-Mart executive said she was surprised with HP's performance. "We are very very pleased with where we are today. We have been very very surprised that they have been able to deliver within this timeframe," she said, adding that she also applauded their attitude. "They would just take every challenge. They kind of delighted in the opportunity."

Although Stewart wouldn't get into details about exactly what HP's servers will be doing for Wal-Mart, she did say that it would involved with RetailLink?the chain's supply chain system for suppliers and customers?with the goal of improving product information access. "What we want is to be able to better present wherever those products are, in the right store in a community. We need to make decisions much more quickly."