The War Chest

MIT’s First-Ever Conference on Digital Experimentation

MIT’s First-Ever Conference on Digital Experimentation

I just got back from MIT’s first ever Conference on Digital Experimentation (#CODECon14).

As one of the only non-academics in attendance (I was invited as the CEO of Alister & Paine Magazine) I found some of the really academic lectures weren’t all that exciting. But there were a two lectures that really stood out. They captured my attention as an entrepreneur, digital practitioner, and amateur economist.

Those lectures were given by Eric Anderson and Sendhil Mullainathan.

Anderson, who is Chair of the Marketing Department at Northwestern’s Kellogg School of Management and Director of the Center for Global Marketing Practice, presented an abstract on Transforming Marketing Analytics in Consumer Focused Organizations. This is my wheelhouse, and I loved every second of it. I reached out to Anderson for slides and hopefully I can update this post later with his presentation, since no recap I can give here will really do it justice (but I’ll try anyway).

In his talk, Anderson discussed how opportunities in marketing analytics are moving much faster than most organizations can handle. As a result, firms have become reactive and tactical.  He explored how firms can adopt a more strategic, customer-focused perspective and said that predictive marketing analytics, driven by rapid experimentation, is central to this transformation. I’ve been using rapid digital experimentation with some of my clients already, so I was pretty happy to have him provide some bias confirmation.

The other lecture which really captured my attention was given by Sendhil Mullainathan.

Mullainathan is a Professor of Economics at Harvard University and author of Scarcity: The New Science of Having Less and How It Defines Our Lives.

He focused his talk on the theory that causation is overstated. As he put it, “To make empirically driven decisions, our methods must reveal causal patterns.  We must know what outcomes each of our choices results in. As a result, experimentation is quickly becoming the tool of choice for decision making. In contrast, it appears that machine learning models because they only produce correlations are problematic for guiding choices. I argue that this is too narrow a perspective on decision-making, that there are many (important) decisions where predictions–with only correlation and no causation–are exactly what are needed for decision making.  Understanding the dual role of prediction and causation in decision making will allow us to better combine machine learning and experimentation.”

As a marketer and amateur econ-nerd, Mullainathan’s lecture really hit a nerve with me. Plus, he’s just a fantastic public speaker. If you have the opportunity to see him speak in person, you absolutely should. He’s funny, energetic, and has all of the entertainment value of a TED Talk but with more substance.