Category Archives: Policy & Regulation

The open data movement has surely gathered momentum across the world. Taking a cue from the United States, which launched its data.gov open data sharing site in May 2009, many national governments have taken similar initiatives to create their own open data sites. These include developed countries such as Australia, Canada, France, Germany, Italy, Netherlands and UK as well as emerging nations such as Brazil, India, Indonesia and Russia. Most of these sites got launched between 2010-2012.

India launched its open data site, data.gov.in September 2012. After starting off slow, it has now picked up momentum and today offers more than 2500 datasets. A dataset is a table of data on a particular area. It could be as large as all the crop production in the country crop-wise, district-wise for the last 30 years or it could be as narrow as exports of a particular item to different global regions in a single year.

The US opened the government data as part of president Obama’s open governance promise, while the first Federal CIO Vivek Kundra, the person behind implementing the initiative, called upon individuals, groups and commercial companies to make use of open data to build innovative apps that would solve citizens’ problems. Kundra consistently championed building apps and even prophesized that in the coming years, there would be “explosion of apps” based on open data.

Since then, these two attributes—transparency and citizen apps—have become the de facto objectives of government open data initiatives across the world. While the developed world has taken to both these objectives, the emerging countries have focused more on the citizen app side, for obvious reasons. Transparency is a very lofty objective to achieve in these countries just by releasing some datasets, when other governance frameworks are not ready.

While both these are worthy expectations to have from government open data initiatives, what is a little worrying is that these objectives have come to define open data priorities and policies in many countries.

Take the Apps expectation, for example. Globally, the role of apps creation from open data has been so overemphasized that many governments try to measure the effectiveness of their open data programs by the number of apps developed on the data made available. That is a completely misplaced expectation because of two reasons. One, data can help in betterment of citizen’s life in many ways beyond apps. Two, it is difficult for governments to track all the apps created. Look at the US data.gov site. Though there are more than 75,000 datasets, there are only around 350 citizen developed apps shared in the site.

Apart from misplaced expectations (and disappointments because of not meeting those expectations), the apps expectation has also resulted in misplaced priorities and policies governing open data.

Here are some of the skewed policies governments have followed because of the overemphasis on the apps part of open data.

Not measuring the efficiency accrued to the economy. Open data initiatives throw important government information in public domain, accessible easily to all. Very often, similar information is separately collected by various others (academic researchers, commercial organizations, other government bodies and agencies) for their requirements, thus duplicating the efforts. In other words, it is inefficient use of time and resources.

Open data, by eliminating—or at least minimizing—the need to duplicate that effort makes the whole economy far more efficient. This is difficult to measure in the short run but over a period of time can be measured. I have never heard any open data evangelist talking about this anywhere.

Further, if the governments realize this, they could cooperate with the other stakeholders and data collection and processing can be optimized to meet the requirements of more stakeholders. In future, the cost can even be shared. This can lead to far more efficient collection and processing of basic information and even enhance data quality.

Limited Outreach. The overemphasis on apps aspect has created a misplaced priority in terms of outreach. The outreach programs of governments in most countries are directed at the tech/app builder community with some tech savvy NGOs/advocacy groups joining in. The entire open data discussion is restricted to these three communities: government, developers, NGOs/advocacy groups. Many major stakeholders such as media, market researchers and academic researchers who could play an important role in showing the latent value that lies in open data are today left out. Even if they do show an interest, they often get scared away by the technical lingo that dominates these discussions. That is a loss for the cause of open data.

In an online conversation hosted by The World Bank on Open Data for Poverty Alleviation, I raised this point. Tim Davies of Practical Participation did agree and had this to say.

I think there is often a failure in open data capacity building to think about the consultants, analysts, researchers and so-on who might be engaged as users of data, and who will provide bespoke value added services on top of it (hopefully realizing social as well as economic value).

Restrictive data formats. Many government agencies implementing open data in their countries focus all their attention on obtaining/creating datasets in machine readable format—a direct result of working from apps backwards. While a lot of time and energy is wasted in conversion/cleaning, a lot of good, structured datasets, that are not in machine readable format never make it to their list of published datasets. That is a big loss.

True, machine readable formats do make life easier for everyone, but ignoring human readable formats is the other extreme. Open data is not defined by any format. Maybe, the implementers of data portals should take some middle path, which will encourage machine readable formats but should not leave out human readable formats such as pdf completely.

Too much emphasis on datasets on consumer interest areas. The overemphasis on citizen apps put an undue pressure on the managers of data portals to work towards obtaining more and more datasets that are directly of interest to end consumers and hence good data to build apps on. So, while a hospital list or a crime info dataset is cheered, a crop production data or exports data is often dismissed as “useless information dumped by government.” While it’s true that data that is of consumer interest can be used to instantly create apps, research on data on agriculture and meteorology, when analyzed at the hands of experts and using right tools can have a far broader and long term impact on the lives of millions of citizens. These analyses could help in maximizing agricultural production/avoiding big disasters/imparting the right skills to unemployed youth and so on, even if they are not created as sleek apps.

Slowly but surely, the constraints of associating open data too much with apps and pre-designed visualizations are being realized. Mike Gurstein, a leading voice about open data argued this in his blog.

But why shouldn’t we think of “open data” as a “service” where the open data rather than being characterized by its “thingness” or its unchangeable quality as a “product”, can be understood as an on-going interactive and iterative process of co-creation between the data supplier and the end-user; where the outcome is as much determined by the needs and interests of the user as by the resources and pre-existing expectations of the data provider?

Though Gurstein’s explicit question is about the rationality of deciding outcomes by the pre-existing expectation of the data provider, the logic can be extended to ask why should it be based on the pre-existing expectation of the apps providers? In most cases, the apps providers do not have too much of extra insight about the end users’ needs.

At the end, it must be pointed out that open data is about making information work for the betterment of society—making lives of citizens convenient, creating the basis for decisions at a macro-economic level, making the economy and business ecosystem more efficient, and yes, minimizing risk. It is not about technology; technology is a very handy tool, though.

Like this:

In September 2011, as the then editor of Dataquest, I wrote an editorial, The Opportunities and Threats of Facebook Democracy. While Dataquest was one of the first publications to do a cover story on how social media was effectively used in the fight against corruption earlier that year (in April), and had celebrated the new power that social media had given to the common people, in this editorial, I had warned against attaching too much importance to the voice emanating from social media by the leaders and policy makers. My reason, of course, was the low penetration of social media. In a large diverse democracy, jumping into conclusion based on what a small section of people belonging to a particular socio-economic section say, was a potentially dangerous and suicidal thing to do, I argued.

The reason I called it Facebook Democracy was that a lot of the campaign by India Against Corruption was actually carried out on Facebook. It was the main mobilization platform.

Since then, Twitter has been used by politicians very effectively to drive their messages. Many politicians and political parties have taken professional help for that purpose. All of us know the power of #pappu and #feku campaigns. While penetration of Twitter is still miniscule compared to the size of Indian electorate, some politicians have managed to have a very large fan following, going up to more than a million. And there are at least ten Indian politicians on Twitter who have more than a lakh followers. Considering that not more than 100 million Indians are online, those numbers are not unimpressive.

Unimpressive they may not be. But as it turns out, most of these followers are fake.

Social media management platform maker, Status People, actually provides a way to check your (and others’) fake followers. I actually checked out the the fake followers of the top ten Indian politicians on Twitter by number of followers and checked how many fake followers they have.

And can you imagine what the average looks like?

It is 59%.

That is, as much as 59% of the followers of these politicians on Twitter are fake. And typically, the bigger the number of followers, the bigger is the percentage of fake followers, though there are small exceptions.

Here is the table.

Politician

Twitter Handle

Total Followers

Fake Followers (%)

Shashi Tharoor

@shashitharoor

1756468

62

Narendtra Modi

@narendramodi

1560092

65

Dr Manmohan Singh

@pmoindia

538323

55

Sushma Swaraj

@sushmaswarajbjp

447766

52

Arvind Kejriwal

@arvindkejriwal

314614

54

Omar Abdullah

@abdullah_omar

274937

54

Subramanian Swamy

@swamy39

165408

42

Ajay Maken

@ajaymaken

151118

55

Derek O Brien

@quizderek

149448

38

Varun Gandhi

@varungandhi80

118728

52

The numbers are as on 1st May 2013

And here are some realities.

Narendra Modi, the potential PM candidate of BJP, heads the list in terms of percentage fake followers, with 65% of his followers being fake.

As many as 8 of the 10 in this list have more fake followers than they have genuine followers. Derek O’ Brien and Subramanian Swamy have the lowest percentage of fake followers in this list.

What Does This Mean?

This, of course, does not suggest that politicians are doing something deliberate to create fake profiles/followers. And since there is not much to choose between different parties, it is not a political statement that one is making. In fact, many politicians themselves will be shocked to know this.

For that matter, there is not too much of a difference between politicians and other celebrities when it comes to the percentage of fake followers. I did check that for a couple of them. In case of Amitabh Bachchan, 73% Twitter followers are fake. For Shah Rukh Khan, that number is 70%. But in case of celebrities, it is a reaching out to the fans, so it does not matter how many fans follow them.

For politicians too, it is a great platform to get their message across, engage with media and at least a certain section of people, who are using this medium. The problem begins, when, their PR managers try to make us believe that they are great leaders because of the large fan following. That is when we get it completely wrong.

In fact, fake followers is just one part. The above platform, Status People, also measures how many of the followers are inactive. For each Twitter profile, it divides the followers into three parts: fake, inactive and good. When you take just those followers that it terms are good (who are real and active), the total followers number drops drastically. Here is the above list of politicians with their “good” followers.

In short, these numbers denote their actual sphere of influence. Except for Tharoor and Modi, these numbers are in thousands; in a country of a billion. And when you combine this to the fact that Twitter reaches only a certain class of people, it follows quite logically that extrapolating the influence/opinion of Twitter to the real world is not a great idea. Not yet.

The UPA government has announced the Direct Cash Transfer (DCT) scheme, with an eye on the elections. As expected, the opposition has cried foul, terming it “bribe” to the voters.

That is politics. And not really the topic of discussion here. In fact, this piece is not even about the economics of it. Economists agree that it is more efficient to pass on the benefits to the deserving directly through cash transfer than indirectly through subsidies. In any case, the government has been talking about it for quite some time. The Budget Speech in the last two years have referred to it explicitly. Many studies internationally have shown that they have, by and large, had a positive effect.

The points that I raise here are not about the politics or economics of DCT but the implementation hurdles that remain. Because even with this limited rollout, it could be the world’s biggest such project. Rushing in to implement may create problems that could shake off people’s confidence on the scheme. This could lead the opponents to project it as a faulty idea per se.

So, here are some of the issues.

1. What about those without the bank accounts? India has less than 25% of people in rural areas, who have access to bank accounts. How will they get the benefit? Does it mean that some of them—those who have bank accounts—will get it and others will not get it? That will be as anarchical as it could be. And the backlash could be severe.

2. How will subsidies and DCT co-exist, even if for a limited period? The government says that the scheme will be fiscal-neutral as it will replace subsidies. Practically, how will that happen, especially in fuel (kerosene), food, and fertilizers? Till all the people are in a position to get the benefit of DCT (today, those who do not have Aadhaar or bank account will not be able to get it), the government cannot touch the public distribution system. Which means it cannot effectively cut subsidies. So, the mechanism has to be in the point of contact (ration stores and the like) to ensure that some beneficiaries do not avail both the benefits, which is next to impossible, as of now. So, the government will continue to run both for the foreseeable future. And surely, it will not be fiscal neutral.

3. What are the alternate channels of supplies? While it is good to say, on paper, that by getting the money directly, the beneficiaries, can opt to buy the products from anywhere, no one is clear what is that anywhere. In many areas, no alternative supply and distribution channel exists. So, how will cash help them?

4. How do you ensure that the money is spent on those products and services for which is intended? How does the government ensure that the money is spent on the products and services that intends to subsidize? In some countries, these subsidies are conditional and are given to women. There is no such plan in India. So, in many families, where men spend a lot of earnings on alcohol and such things, more cash means more money to get drunk. The possibility is very real in India.

The issues raised here are not meant to argue against the implementation of DCT.

But the fact remains that changing the entire subsidy regime requires a lot of thought and preparation. The government started on the right note by appointing a task force to suggest the ways and means of implementing this.

The task force, headed by Nandan Nilekani, Chairman, UIDAI, submitted a detailed report, recommending creation of what it called a Core Subsidy Management System (CSMS) to implement the new subsidy regime. The task force foresaw the gap that exists in the payment system reach and recommended this

Since it may take a while for the payment systems in the country to gear up for direct transfer of subsidies, an intermediate step may be considered where the subsidy difference is transferred to wholesalers/retailers in the first phase, and only later on to customers.

But the government has disregarded it and has announced DCT right away. Also, there is no news on where the rollout of CSMS has reached.

With all its good intentions, the government will have only itself to blame, if the whole idea backfires.

The much-awaited policy decision to allow foreign direct investment (FDI) in multi-brand retail has become the most high profile reforms measure announced by the UPA government. It has not just been a booster shot for the UPA government—which seemed to be doing nothing right just a few weeks back—it has shifted the debate from corruption and policy paralysis to reforms, with businesses and middle class now finding themselves on the side of the government.

But what is the issue about? In short, those who support it argue that it will not only give consumers more choice (and hence more power), it will fetch better prices for farmers. Those who oppose it primarily argue that it will kill the small traders—the kirana store owners. So, the debate has essentially become one with small traders’ interest on one side and farmers’ (and to a smaller extent consumers’) interest on the other.

But here, we are missing out a really, really big point.

Wall-Mart is the largest civilian employer in the world, with more than 2 million employees. Tesco is the largest private sector employer in the UK. Woolworths and Wesfarmers are the two largest employers in Australia. Carrefour is one of the largest employers in the world, though because of its distributed business, it is not the largest employer in its home country, France. Sears, Target, Home Depot all feature among the largest employers in the US.

Wall-Mart employs more than two million people globally. In the US alone, it has 1.4 million employees—that is a little less than half a percent of the total US population. Tesco employs more than 500,000 people and Carrefour some 475,000 people. Both Woolworths and Wesfarmers in Australia employ more than 200,000 people each. Together, that is a little less than two percent of Australian population.

And how many employees does Futures Group—the largest retailer in India, a country of 1.3 billion people—employ? Just about 35,000 including its insurance and other businesses.

If you consider this aspect, there is not much to debate. The employment generation potential of organized retail sector is immense. And potentially widespread.

When IT came as a big bang service industry to India, it created a big employment opportunities. But that was restricted to a certain section of the society—the engineering graduates. And it created jobs in a few locations. The BPO industrt democratized it by providing opportunities to graduates, took the action to tier two cities, and reduced the time to impart right skill to these people to make them productive. Retail is the next logical wave. It will further democratize the organized services sector by creating the jobs for those who have had some high school education, who can speak local language and maybe have some working knowledge of English. And the time to provide skill training to make these people productive reduces further.

The case of big box retail, hence, is justified, looking at it purely from an employment generation perspective. It is sad our politicians and public commentators are missing the point.

Whether it is a desperate attempt to secure a place in history or a genuine attempt to break policy paralysis, the Cabinet has taken the bold decision to allow 51% FDI in multi-brand retail, albeit with a provision that state governments would decide if they would like to allow it in their states.

Though many see this as a compromise for somehow moving ahead, one feels it is a masterstroke.

Firstly, it suddenly takes away the legitimacy from the opposition to FDI in retail by CMs like Mamata Banerjee and Narendra Modi. An average citizen of Delhi or Mumbai, who wants global brands in his city, is bound to ask, who is she to come in the way of our access to the global retail outlets?

Secondly, if the implementation happens well, soon the citizens of states that have not allowed FDI will see the difference it makes, as they visit cities in other states with such outlets. It will be difficult to resist the “middle class” pressure for the governments then. Imagine, for example, in the National Capital region, Gurgaon having all the big global retail brands, with Noida not having a single one of them!

Finally, if the government and the supporters of FDI in retail, play it well, it should be sold to citizens as a farmers-friendly rather than large business-friendly policy which it actually is. With the farmers and the middle class supporting it, it will only be the small traders who will be opposing it. While they are a powerful community in states like Gujarat, UP, and Tamil Nadu, states like Karnataka, Odisha, and Bihar will not find any strong reason to oppose FDI. Most of India’s potential locations, such as Delhi, Mumbai, Gurgaon, Pune, Hyderabad, and Jaipur will have the new brands. The large cities that will be left out will be Chennai, Bangalore, Ahmedabad, and Kolkata. Out of which, it will be interesting to see how things unfold in Bangalore, as the state has no logical reason to oppose it.

But the most interesting thing to watch will be e-commerce. Initially, the policy was vague about e-commerce. But in April this year, the government clarified that all the rules that are applicable to offline retail would be applicable to e-commerce as well. This clearly meant that all the plans of companies like Amazon had to be shelved. With the new policy change, they can enter in India. So, expect a new era altogether in e-commerce. Good luck, Flipkart!

But interesting will be to see how offline retail brands such as Wal-Mart or Tesco unfold their India strategy in this policy regime? Access to the top two cities and some of the other biggest markets will surely make India entry attractive. But once they enter and build their supply chain, especially the procurement network, there is nothing that is stopping them from selling online to the entire Indian population, irrespective of where the buyers are located. They will not violate any law as they will not have to open any “outlet” in those states.

Question is: will that happen? Will the politicians still not try to hound them? Or as many optimists hope, all this is meaningless discussion, as soon, most states will open up FDI in multi-brand retail.

In either case, a vibrant, more competitive retail market has implications for the e-commerce market.

The stated advantages, as envisioned by the policy, include maximization of use of data, avoidance/minimization of duplication of efforts on collection, facilitating integration by leading to common standards, providing ownership information, faster and better decision making, and of course, equitable access to information by all citizens.

Ever since the then Federal CIO of the United States started in May 2009, many counties have launched similar sites. India is the latest country to join the bandwagon.

The data.gov.in site has debuted with 13 raw datasets provided by seven departments and four apps provided by four departments. As part of the plan, data management offices are being created in each of the departments to be headed by a senior official called data controllers. Five ministries/departments have already identified their data controllers, whose names are available in the site. These are Department of Public Enterprises under the Ministry of Heavy Industries & Public Enterprises, Department of Disinvestment under Ministry of Finance, Department of Fertilizers under Ministry of Chemicals & Fertilizers, Ministry of Food Processing Industries, and Ministry of Micro Small and Medium Enterprises. The departments will be responsible for uploading the datasets directly, for which NIC is helping them by providing training and technical help. It is conducting a workshop on 5th October focused on this.

India, however, is not a part of open government partnership, a consortium of more than 50 countries. The initiative was started by nine countries, including India, but India withdrew just before launch. India was reportedly “concerned about the Independent Review Mechanism” which opens participating countries for reviews by outsiders.

However, India has been supporting the open government community by helping create what is called an open government platform—an easy-to-use toolbox that allows smaller countries to go for similar portals without worrying about technical challenges. The platform was launched in last March.

The launch of data.gov.in marks a new chapter in governance. It is a pity that it has got almost no mention in the media, especially when corruption and the role of public institutions are being debated so intensely.

In the US, the opening up of government data to public has seen innovative applications being created by third party organizations using the data (maximizing use).

Many say a more laudable goal in India would be the avoidance of duplication of efforts and resources in collecting data. This, however, is a lofty expectation to have from a transparency initiative like this, because it is not lack of availability/knowledge but personal ego battles and/or lack of coordination between departments that are the reasons for this duplication of efforts and resources. A recent example is the tussle between Home Ministry and the UIDAI on collecting data for National Population Register and Aadhaar.

But the private sector, academics researchers, and NGOs/advocacies can surely benefit from getting easy and timely access to government collected data. With analytics and data vizulazation becoming the hottest areas in technology, an initiative like this could not have been more timely.

Politically speaking, the decision by Anna Hazare to disband Team Anna and take up the fight through electoral politics is both good news and bad. Good because it upholds the supremacy of electoral democracy—which was being pooh poohed by the likes of Arvind Kejriwal just a few months back. But what I feel bad about—and I am not a supporter of M/s Bedi and Kejriwal—is that an experiment to take an alternate route has failed. Yes, despite the power of Facebook and Twitter to support it this time around, as many of us never forgot to add. But as I have pointed out many times, including in this blog (Digital divide is now political…), reach of social media—or for that matter Internet—is too limited to fight a successful battle against the government and the system.

So, do I mean to say that we have little hope—as long as we have a democracy and as long as we, as a people, are not completely honest, we will have to tolerate this large scale, systematized corruption?

Not necessarily. But if we have to really find a solution to the problem of corruption, it has to be taking a different approach that fundamentally changes certain key parameters, not taking the same path again and again.

The big proposed entity called Lokpal—that is thought to be the panacea for all ills by Anna and his supporters—is nothing but yet another costly addition to the already overburdened system. If the Legislature, Judiciary, the Executive and the Press could not do it, by what logic do we expect yet another new body with its base in Delhi to eradicate corruption? The demand has been for more power to it. There is no logic given by anyone how more power itself will translate into more effectiveness in checking corruption. After all, the members would be people from amongst us. Why should one believe that they would be more honest than you and me—and our politicians and bureaucrats?

The problem is we are seeking a solution in the old, centralized model with a set of people having absolute power over everyone. Just that instead of being called MPs or ministers or secretaries or editors, they would be called Lokpals.

What if we take a fundamentally new approach? Instead of trying to check corruption by instilling fear of punishment after the act of corruption is done, what if we can ensure that corruption is minimized by making it more difficult to do it, i.e instilling fear of getting caught while doing it. That should be done by bringing in transparency.

Such transparency is possible only when there is easier access to information by a wider section, ideally members of the general public.

Two fundamental principles are cornerstones of this approach: one, instead of centralized systems, we go for decentralization; and two, instead of giving real power to people, we give it to a computer system.

Decentralization does not necessarily mean chaos. Wikipedia—despite whatever limitations it may have—has shown us how the collective power of people can be credible and dynamic at the same time. But to ensure that we prevent mobocracy and chaos, there has to be defined rules and processes (as in Wikipedia) and massive information infrastructure to store, forward, and process information. That requires a powerful (ideally distributed) computer system.

As everything becomes available to a wide set of people, the system will ensure that few dare to indulge in violating the rules. A person may not fear another person; but everyone fears the public.

We have seen that happening in cricket. The third umpire—though there is a person whose name is associated with it—is actually a computer. The replay is on a huge screen for the world to see. And technology ensures that there is no intended wrong decision. The same principle will work here.

In a technology-enabled system, the information itself will have the power to make everyone exercise restraint. A huge computing plaform—lets say a supercomputer—can, on a continuous basis, monitor for exceptions. There can also be ways and means to lodge anonymous complaints by the whistle blowers. Initially, people may misuse it to trouble opponents, but soon, the system will take care of itself. If the processes and technology are good, a false complaint will result in calling out the bluff. By moving from an investigation mode to a prevention mode, the system itself will become more “less corrupt”. There will be experts, advisors, information analysts—from any walk of life. But the power will not lie with them; it will lie with the computer in particular and the whole system in general. The system would be fault tolerant and designed to learn from experience.

Of course, any such system can be effective only when there is a lot of information generated electronically. That means a lot of government processes need to be automated. Thankfully, that is increasingly happening. That will supplement the reactive mechanism of the Wiki model by a proactive check on processes and exceptions. In such a scenario, RTI would be seamless and would be like a Google search.

Instead of instilling fear of punishment after the corrupt act is done, it would instill fear of exposure while doing corruption. So, not only would one get caught but would get caught before he/she gets any benefit out of that.

Such a system will ensure the following

Any exception is caught and reported, almost in real time

All information is stored in multiple locations so if something that cannot be brought to public notice in real time because of sensitiveness of the issue, they can be exposed in future by the system. Remember WikiLeaks?

Ensure speed and efficiency in addition to transparency

By making all the citizens participate, we would give collective responsibility to everyone while rigorous processes with technology underneath ensuring that there is no chaos because of that. This will still not be able to eliminate corruption but will make it far more difficult to do corruption, thereby significantly reducing it.