In my previous article “India is ready for being a data driven society”, I discussed about data being captured, data yet to be captured, and about making captured data more visible. This article tries to explain why I think big data is different, how big data should be approached, and who can become the key pillars of big data!

It is no secret that innovation has always happened in small organizations. Large enterprises acquire such innovations by paying a large sum of money. Big data, however, is an exception. Let me explain why.

Governments are capturing public data. So obviously, without the participation of government departments, any innovation in big data will not be complete. Small organizations/startup’s will become disadvantaged here as they have been traditionally good with great ideas and executing those great ideas but lack experience in engaging with governments. Psychologically, they do not feel comfortable dealing with governments due to a larger perception of slow decision making, larger bureaucracy, red tapism, and other reasons best known to everyone. So the chances of innovation, therefore, with big data will reduce dramatically. It looks like big data or any innovation with big data is left for large enterprises creating barriers to entry for small players. This is the reason I say big data is different business.

There is a new trend now in the ecosystem. Large companies are trying to approach small companies and promising to support their innovation, but small companies are wondering whether or not they should accept the support as there lies the hidden risk of the larger companies proclaiming the idea as theirs – unless it’s a patent pending technology. This trust deficit is affecting the progress.

I have had an opportunity to volunteer for a social organization called Techforseva. I was super excited about what they are doing. They are getting NGO’s, Corporates and Experts/Scientists on to one platform. The idea behind this initiative is to provide funding from CSR initiatives to NGO’s who really deserve capital and also provide technical, marketing, or any other help that the NGO’s lack and fill that gap from experts/scientific community contributions. After I spent some time at Techforseva I understood something new about the “NGO’s/Social organizations” that I had ignored before. After I interacted with a few NGO’s, I was surprised to see that so much of data held by them was untapped or unexplored – and it was a real data goldmine. It was pretty clear that without a full-fledged participation of NGO’s/Social Organizations, big data will not complete a full circle.

Following are what I think constitute the four pillars of big data

Media does not just print stories, but is also a hub for big data as it holds a huge data of text, paper, audio and video archives, and is recording life as it happens. With the digital television revolution, they also have the data about viewership on various topics that the billions worth advertising industry are leveraging. This data can also be tapped for various other purposes.

With the active collaboration of these four pillars, big data can impact the way life is conducted on the planet in a more intelligent manner, fully optimizing the human potential.

Please do not hesitate to be more vocal, ask questions, leave comments, like and share if something is useful to you. Your participation will encourage me to write more to help you understand the various dimensions of big data.

Stay tuned for my next article – “Big data, Agriculture and Indian farmer”