It's Time to Ditch the Spaghetti Diagrams

Todd Michaud spent years leading retail technology teams for Dunkin' Donuts and Baskin-Robbins and today serves as the VP of IT for a billion-dollar franchise restaurant company. He also runs Power Thinking Media, which helps restaurants and retailers with social and mobile challenges.

With all of the new data coming in from mobile and social, retail IT has a truly strategic psychological problem. The old way of creating interfaces between systems can't scale and will not deliver the results this new world of information overload demands. You've got to stop thinking about interfaces and start thinking about services. You've got to stop thinking about batch ETL processing and start thinking about real-time data integration.

You've got to start accepting cloud computing as a method of scaling your computing platform up and down. You've got to dive deep into the world of big data (and unstructured data). In short, you've got to rip out most of your information architecture and start over.

The challenge of deriving true business value out of the wealth of data created within a retail organization is not a new one; just ask someone who works in Business Intelligence for a living. And although there has been a lot of progress made in this area over the past 10 years, in the majority of retailers that I speak to, their ability to extract meaningful information from their current data is both limited and time-consuming. It is often done with myriad interfaces, across a patchwork of systems, and requires spaghetti diagrams to explain.

In the past few weeks I have had a chance to talk with Dave Levitt, VP of IT for The Children's Place, about how the rapidly increasing needs for sharing data across various back-office platforms was quickly dragging the IT team into interface hell. Add to that the marketing team's desire to fine-tune their marketing messages with data from social networks, and you've got an IT team who spends most of their time collecting and moving data around.

Instead of taking the easy way out and asking for more programmers or DBAs, Dave and his team put together a robust information architecture using a trio of technologies that, until recently, wouldn't be caught dead together in the same enterprise: an enterprise service bus, cloud computing and open-source software.

But with these technologies in place, The Children's Place is able to maximize E-mail marketing by customizing 80 percent or more of its E-mails to be highly tailored specifically for individual customers. More importantly to the IT team, they now have a highly flexible, easily scalable system that has enabled their team to focus on higher value activities than babysitting data.

Other chains should take note, because most IT teams are ill-equipped to face the tsunami of data heading their way as a result of social media and mobile technology. Add to that the complexity of cloud-based applications that are invading the enterprise and seemingly always need access to data from within the datacenter. Then, if you keep trying what you've always done—the definition of insanity—you are going to find yourself all wet.

But with platforms such as social media and mobile technologies dramatically increasing the amount of data being created, the larger need will be to gain insight from this data. I call it the Information Revolution, and it's already underway. The Information Revolution will consist of three phases: "help me" (decision support); "help you" (marketing optimization); and "help me, help you" (customer optimization). Retail IT teams need to start putting their boots on and getting their hands dirty right now, before it is too late.

Let me explain a little bit about these phases and why they are important to the business.Let me explain a little bit about these phases and why they are important to the business. The help me phase will be when retailers leverage social and mobile data to make better business decisions. Think of this as focus groups 2.0. This phase of the Information Revolution has already begun. Companies are mining social and mobile data looking for brand sentiment and, in some cases, engaging with their customers. But in my experience looking at most of the Facebook pages and Twitter feeds, most retailers do not understand the word "engagement" (but that is for another column).

The help you phase is all about marketing optimization. This is where The Children's Place is. The company is using its business data, combined with social data, to enhance marketing efforts within its E-mail platform. The Children's Place hired some really smart mathematicians from MIT to create algorithms that best optimize the offers for its customers. The company hopes to get from 80 percent to 100 percent customized.

The final phase of Customer Optimization, the help me, help you phase, is the ultimate nirvana. In this phase, retailers will not have loyalty programs, E-mail marketing programs and online merchandizing programs. Instead, each customer (or potential customer) will have a unique program built for them. This is taking personalization to a whole new level. If a customer never opens an E-mail offer but responds to a Facebook or Twitter update, you will stop sending those people E-mails. The Web site and mobile interface will be customized to drive a behavior, such as moving a browser to a buyer or increasing frequency.

If you think this is farfetched, I suggest you call some people in the casino marketing business and ask them about the evolution of their marketing strategy.

Why is this important? Because as the marketing opportunities for retailers continue to splinter, and the signal-to-noise ratio continues to get worse, retailers will find themselves quickly "competing on relationship." When costs and convenience are neutralized, people will buy from the retailers that they have the best relationship with and where they have the best experience. If you haven't extracted meaningful insight from your customers' social and mobile data to better engage with them or to create a better experience for them, someone else will

So what does that mean for the guy or gal who is responsible for moving all of that data around? First, I think you'll laugh every time someone mentions that "social media is free," because social data is about to become the biggest and most expensive project you have ever done.

You need to learn about open-source projects like Apache Hadoop and what it means to process large amounts of unstructured data. You need to understand the mechanics of social CRM and how your company can use this type of tool to operationalize marketing data into actions in the store and online.

You need to create a new information architecture that uses an enterprise service bus to create services to move data, rather than interfaces. Think about data being "published" and "subscribed to." You need to create a plan to migrate and retire your existing interfaces. This becomes critical, because your marketing team doesn't yet know what to ask for, so you need to build a system that is designed to help them answer questions they don't yet know they want to ask.

You need to imagine a time where you spin up a thousand virtual servers to process a nightly batch, only to wind them down again the next day. And that night is closer than you think).

What do you think? If you disagree (or even, heaven forbid, agree), please comment below or send me a private message. Or check out the Twitter discussion on @todd_michaud.