May 2018
« Apr    

N = all ….An Impending Big Data Era

Sanjay Jaju, Secretary-Information Technology and Communications Department. Government of Andhra Pradesh

Sanjay Jaju, Former Secretary-Information Technology and Communications Department. Government of Andhra Pradesh

By Sanjay Jaju

While I force my fingers on the keypad of my laptop typing this piece, I can’t remain oblivious to the fact that while doing so I am myself contributing my share of kilobytes to the already overcrowded world of virtual real estate.  It simply baffles me to note that the data exposure a human mind has in a day on an average in present times is equal to what our ancestors had in their entire life time, say 5 centuries back. Research avers that more than half of the data available now has been uploaded on the net in the last six months and these trends would become sharper and grow exponentially as time moves on.

We are living in very different and interesting times in an age marked by Data explosion.  Gone are the days when we used to work on small samples to work on our hypothesis.  Neither could we have the resources to rummage through the bigger numbers nor did we have the tools and computing power to derive meaningful insights from such numbers.  All that is a passé now, we have acquired capabilities that bring computing power in pocketsize phones and big servers that can mine data unthinkable in the past.  Age of Big Data has arrived and is likely to shape our thinking and our decision-making in an unprecedented manner, never seen before.

Do we know that every little act of ours be on the phone or on the social network is now getting datafied somewhere and is available for researchers and marketers to derive conclusions and correlations.  While in the past, all our research efforts and predictions were guided by our intention to decipher cause and effect relationships between mutually interacting variables, It’s now the correlation and the regression that researchers are looking for, sitting as they are on a huge stockpile of data unthinkable in the past.  The decisions that were intuitive or based on hunches now rely on data and the analytics around.

The question then is are we heading towards a Data driven society and if yes, what would be its impact on our daily lives, our relationship with the state and vice versa.  This piece intends to address some of these questions.

It’s indeed true that every moment that we spend on the net may though appear purposeless and facile, but to an intelligent algorithm on a machine far way, it turns out to be a goldmine of information. The algorithm and the third eye on the net is actually able to go to the extent of figuring out what our next moves would be by analyzing our behavior, habits, likes, dislikes, the footprints of which we leave consciously or inadvertently on the net.  Beauty truly has started to lie in the eyes of the beholder.

It has more or less become a fetish now to google for answers to any question personal or professional that crop in my mind.  Many of us even look at Google to gauge what is trending hot.  The outbreak of the winter flue predicted by Google in United States down to specific regions is a case in point.

The question is whether all this can start providing solutions to big research questions like decoding of the genetics or the esoteric particle physics or provide economically optimal solutions to the pressing needs of the consumers.

If one looks at the way decisions are made in the government, the need for a better decision support system doesn’t look exaggerated.  Big Data can become a pivot for such a developement.  Governments sit on huge datasets, the only problem is it being ensnared in departmental silos with minimal mutual interactions and therefore is of no use.  If all this can be integrated through a common platform, meaningful insights can be drawn which would allow better decision making based on scientific rigors and analysis.

Aadhaar (UID) numbers intend to bridge this gap at least for citizen databases in India by acting as a unique string across heterogeneous datasets making them talk to each other and talk coherently for conclusions and inferences to be drawn.

The question that arises then is would this tantamount to snooping or meddling into the privacy of the individuals.  The critics would argue that every infringement of private rights is justified by its negative impact and fall out on collective societal rights, if not done so.  Big data age is surely going to make this distinction become blurred and keep this age-old debate between Privacy and Public good alive and kicking.  This when blended with added complexities of the modern world with rising aspirations and mounting challenges of maintaining the law and order and the incapacity of state to meet such aspirations and challenges makes the debate veer towards big data and since such capacities can be raised by using Big Data, would the State sit back, ignore and remain blind to such milking opportunities. The question is whether the states would be ready to jeopardize public interests for the sake of protecting individual rights.

The flip side and risks associated with Big data is its ability to draw conclusions based on correlations and presumptions.  It is quite likely that Big Data may allow the State to pinpoint and know with a degree of certitude, the occurrence of an event much before its real happening.  In such a case, can it be presumed that the State, which is duty bound to prevent such occurrence, not act against the so-called prospective perpetrator.  If it acts, it would be based on a presumption rather than occurrence of the event itself.  Would it then meet the ends of Justice, which relies heavily on Natural laws wherein it’s presumed that somebody is innocent till it is proved otherwise and that too beyond any reasonable doubt.  The scenes from the movie, ‘Minority Report’ immediately comes to our mind and days are not far when we start seeing such happenings thanks to Big Data and ability of the system to adduce inferences based on patterns and thoughts which until now were in the preserves of the narrow confines of our minds.

The logical corollary is what if there are machines which makes such intelligence available to individuals, will it not take the sheen away from a very interesting though deceitful world full of emotions, back stabbing, treachery, information asymmetries and opacity that separates and clouds our inwards to outwards. What if we had the first hand information about everything way in advance, will it not kill the happy excitement that shapes our efforts, in a sense our entire life journey and all its associated toils or sink us for ever by its premonitions and predictions about the impending doom.

The effects of this are already palpable, what shape will it take in future is anybody’s guess.  One thing that’s certain is that it would irreversibly change the way we deal with each other much more than what could be done by the Social networks and media.  Internet has developed based on an economic order dictated by commerce around information, its going to surely acquire a new hue in this new big data age. It would therefore be sagacious to flow with the tide and master the current rather than struggling to flow against and be certain to fail.  The choice is obvious.

About the Author:

Sanjay Jaju comes from Indian Administrative Service and is the Secretary to Government of Andhra Pradesh, Information Technology & Communications Department. He has been an ardent advocate of Information Technology for Development and Governance.  He has worked in diverse areas like Urban, Infrastructure, Civil Supplies, Education & Tribal Welfare.