The Fallacy of Interactive Media Business Intelligence
By Ramin Shokrizade
2-27-2012
As corporations get larger, and the population groups they server also get larger, there has been a move away from direct overt survey techniques and a preference for covert data mining. This makes sense, because even if you did have the resources to survey a million customers, and 10% of them responded, the time it would take to carefully read and analyze 100,000 survey responses would be prohibitive.

Data mining can produce very reliable data in real time without the human bias a survey reader might introduce. It also mitigates the bias inherent when surveyed individuals know they are being surveyed. Once you have the data, you then can apply various regression analysis techniques to make predictions based on that data. This works as long as you are aware of all of the significant contributors to the behavior you are trying to predict, and have measured same.

The danger comes when you attempt regression analysis without knowing all of the relevant factors. The peril is especially high if you don’t know all the variables, but convince yourself that you do. Under this condition, when the data works the way you anticipated, you say the model is good. When it does not, you blame “consumer randomness” without admitting the possibility that the model is insufficient.

Decision makers in interactive media, and other industries seeking to tap the same markets through gamification, are increasingly falling prey to this fallacy. They assume the models that work in real-space will also work in virtual space. To get better data, they employ ever more powerful data mining mechanisms and the analytics personnel required to interpret that data. The problem with all of this content is that without the proper context to place this data, it becomes at best useless, and potentially quite misleading.

As misleading data continues to lead to industry losses, our leaders react in a manner very similar to how Agent Smith reacts to the enigma, Neo, in the third Matrix movie: “We need more”. This is not an effective method of dealing with problems you do not understand. It isn’t even an option at some point because the number of people in the labor pool trained to provide this service is limited, and the quality of these individuals hired will go down over time as the best candidates are hired first.

As with any addiction, the first step is to admit you have a problem. This can be an insurmountable hurdle for many business leaders. The second step, for those that get that far, is to seek solutions. Where to look? You can’t look if you don’t even know what you are looking for. If the solutions exist completely outside of current business theory, then those operating within the current business framework will be entirely unequipped to find them.

It is ironic that during a period of unprecedented consumer demand for our products, and ever-lowering barriers to entry for producers, the failure rate in this industry just seems to rise. How long will industry be willing to endure this situation before adapting? I, personally, am beginning to tire a bit of the blame being put on the consumer.