Data, Guts and Innovation

Pavan Soni, Founder and Innovation Evangelist, Inflexion Point Consulting Bangalore based Inflexion Point, the consultancy firm scientifically assists businesses, at any size, fail owing to an overt and brings science into ways of running businesses.

With the pervasive and affordable information technology, the access to data analytics is gone far beyond the realm of a few large organizations. Startups and consumers, alike, are enjoying an unprecedented access to data and, in some cases, intelligence, to an extent that they are often at a loss as to what do with all this. One of the ways to put the power of data to use is to listen to the customers more intently, scan the rapidly evolving technology landscape more closely, and reach out to a wider set of investors, both institutional and the crowd. All this is meant to bring about innovation, hopefully, more rapidly and cost effectively.

However, the nature of innovation is not so straightforward. History has shown that several breakthroughs have resulted, surprisingly, from not listening to the customers, and not chasing the investors, or by taming the technology trends, but by doing the exact opposite. These are serendipitous inventions, or as Steven Johnson calls 'slow hunch'. Past is replete with instances of products and services emerging from someone’s gut and the labs, rather than from a deep interest in the market unfolding and consumer insights. The Harvard Professor, Clayton Christensen, calls this as the ‘innovator’s dilemma’- which is, whether to listen to the current customer or go for the non-customers with a new, disruptive technology.

In this article, I attempt to put the power of data analytics and the might of gut feel at their rightful place, and contend that analytics can, at best, result in incremental innovations, but for the radical and the breakthrough types, the gut still leads decision making.

Steve Jobs was famous for conducting no market research and not hiring consultants to advise him on the market pulse or which technology to go after. His entry into the music industry and then into mobile phones squarely goes against the management dictum of 'core competence' or ‘sticking to the knitting’, and yet Apple disrupted industry after industry. The case of Nestle launching Nespresso, a marked shift in both product and business model, against an overwhelming market-led advice of not to, is another case in point of how 'listening' to the market may not always be prudent. And then there are a host of technology breakthroughs that the likes of James Dyson and Dean Kamen keep surprising the audience with, with an effect no short of magic. But we all understand that the moments created by these geniuses, including Jeff Bezos and Elon Musk, are few and far between.

If we broadly identify the taxonomy of innovation, one could think of innovations as
being sustaining or disruptive in nature. While the sustaining innovations target the same customers with similar or improved technologies, the disruptive innovations open up new customer segments, hitherto unserved, by adopting a new technology or a business model. Based on the intensity of the impact, the sustaining innovations can further be classified into incremental and radical ones. If you think of a performance vs time curve, the incremental innovations would be indicated by upward rising curve, while a radial innovation would look like a step-function, or a leap.

" Data driven insight is a great way to arrive at incremental suggestions to keep pace with the market and evolving customer taste, but a one-off break through comes straight from the gut"

Incremental innovations are best served by analytics, while radical innovations emerge from gut feels.
Figure 1: Incremental versus Radical innovations

The nature of analytics, more so with the advent of Big Data and Artificial Intelligence, is to draw inferences or patterns from a large pool of data. This underlying data could pertain to consumers’ spending behaviours, time spent on various activities, like commute, at office, at home, etc., or on what psychological nudges work in changing consumer behaviour. The very scheme of pattern recognition is that it discounts the outliers, or the low occurrence events, in order to come up with a generalizable finding. How do these inferences translate into innovation? The companies adopt such data driven insights to improve the existing products, processes and services, often marginally, to meet the customer’s stated and unstated requirements. Adding more comfort features to the car, or more applications on the mobile phone, amongst others, are good examples of the incremental innovations, which have no reasons to fail.

Such an incremental approach has two severe limitations. Firstly, most changes emerging from such an incremental approach are marginal and do not offer much competitive advantage to the incumbent. Today’s good-to-have becomes tomorrow’s must-have, and so on. The second, and more dangerous limitation is that this data driven incremental innovation approach takes the eyes off from the real outliers and disruptions that have the potential of shifting the entire landscape. Which is to say that while the firm is busy fighting the battle, they have already lost the war.

The case of one of world’s largest CD/ DVD maker- Moser Baer-is very pertinent here. While this Indian company did very well in bringing down the prices and improving the quality and durability of the optical storage media, the market itself shifted from storage as a means of consumption to streaming! How many of us buys DVDs anymore? Entertainment is mostly streamed into our devices at the comfort of our living rooms. That’s where a data driven, incremental approach doesn’t help uncover blind spots.

The firm needs such an incremental approach punctuated with radical innovations which are driven by the gut feels or the possibilities of technologies, and not so much about what's customer saying.
If you ever were to reach to a customer with the classic keypad phone before the iPhone days, a customer would have, at best, offered ideas around smaller size, lower weight, more colours, better battery life et al.,and not that 'let's get rid of all these keys’. It took a push of some technology and a leap of faith to bring about a radical new way of thinking of phones. And now, it's become a dominant standard.

I am not suggesting you dismiss what data has to offer, but to dismiss one’s gut feel isn’t wise either. Data driven insight is a great way to arrive at incremental suggestions to keep pace with the market and evolving customer taste, but a one-off breakthrough comes straight from the gut.