Opening a Can of Worms: Government and Climate Change Data
Wednesday, December 2, 2009
Since open government is a major initiative of the administration, and so is climate change, one mighty collision is coming soon.
The day after his investiture, President Obama issued a memorandum to agency heads on transparency and open government, pledging “an unprecedented level of openness in government” to promote “transparency, public participation, and collaboration” and “strengthen our democracy and promote efficiency and effectiveness in government.”
The Office of Science and Technology Policy (OSTP) was given the lead for this effort, with the creation of new positions of Chief Technology Officer and a Deputy CTO for Openness. The latter job is filled by Beth Noveck, New York Law School professor, initiator of the interesting and creative Peer-to-Patent program, and author of a book Wiki Government: How Technology Can Make Government Better, Democracy Stronger, and Citizens More Powerful (2009).
“Openness” is a big topic. The Obama-ites, in their usual solipsism, assume the world began anew last January. But the effort to open up government has been an obsession of U.S. politics forever, with repeated Freedom of Information Acts, Sunshine Acts, Disclosure Acts, and so on. Much of the current drive for openness continues these efforts and adds computer tools, or is simply the use of the computer and the Internet to improve governmental interactions with the public. A better Internal Revenue Service or Transportation Security Administration website is indeed a public benefit, but it does not represent any serious change in government.
A little-understood dimension of the climate change story is how small is the number of scientists who occupy its core—the supposed ‘thousands’ distill down to a few dozen, at most.
In one area, though, openness is a big deal indeed—the area called crowdsourcing, or, more prosaically, the decreased transaction costs of community interaction.
And, as chance would have it (or perhaps God does have a sense of humor), we are getting a spectacular demonstration of the power and importance of crowdsourcing from climate change data.
A major barrier to producing public goods that lack the support of any revenue stream—including that of monitoring government data and policies—has always been the high cost of organization. Such activities rely on the contribution of fragments of time and expertise by people who have some other means of support, and it is difficult to find such people, bring them up to speed, keep them in communication, and make use of their efforts because these various overhead costs eat up the available resources.
Furthermore, pre-Internet, the communication problem was insurmountable. Trying to coordinate an amorphous group of casual participants by phone and mail is not doable. The classic telephone tree of the volunteer organization was about as sophisticated as one could get.
The Obama-ites assume the world began anew last January, but the effort to open up government has been an obsession of U.S. politics forever.
The Internet changes this by creating possibilities to combine the efforts of numerous geographically dispersed people who select themselves and willingly contribute limited amounts of effort diverted from their main occupations. If external motivation exists—whether academic, social, political, religious, or intellectual—and participants need not worry about getting paid for their efforts, then opening the gates of communication, enabling self-selection and self-organization, and stepping back to watch evolution in action can produce spectacular results.
One of the first to articulate this transformational power was Linus Torvalds, founder of the Linux movement, who has noted that Linux could not have been created without this power of the Internet. The model does not really apply to long-term, heavy-duty production of serious commercial products, and Linux has long since been professionalized into a well-funded collaborative of computer hardware and service providers, closely controlled by a group of insiders; but for projects that do not need commercial motivation the basic idea is powerful indeed.
This fundamental insight of the tech world about the possibilities of crowdsourcing was an important component of the pressure that resulted in the administration’s open government initiative (see Open Data is Civic Capital: Best Practices for "Open Government Data").
Scientists who have supported their fellows because it is impolite to challenge a fellow in his or her area of expertise must weigh the price of solidarity in terms of the damage done to science as an institution.
One quick action to promote the initiative was the creation of Data.gov, designed “to increase public access to high value, machine readable datasets generated by the executive branch of the federal government.” The idea is to make government datasets available and manipulable and let the tech community do its thing to produce value, and to leave it up to individuals to determine what that value might be.
In reaction, intermediaries are already organizing the information to make it even easier for programmers to play with—see the Sunlight Foundation and the Microsoft Open Government Data Initiative. Of course, another purpose is to monitor government itself. One thing on which people of all political persuasions seem to agree is that their opponents wield undue influence, and this must be exposed to the disinfectant of sunlight.
Since open government is a major initiative of the administration, and so is climate change, one mighty collision is coming, because the manner in which the climate change researchers have handled their data is absolutely antithetical to the principles of open government. Information has been kept secret and the processes by which it has been “adjusted” have remained opaque, and, when revealed, turned out to be incomprehensible or dishonest.
It is not the emails that are the main issue, but the underlying documents and the computer code. To take a source not in “the Denier” community, Eric Raymond, guru of the Open Source Software movement and author of the urtext of openness The Cathedral and the Bazaar, says:
This, people, is blatant data-cooking, with no pretense otherwise. It flattens a period of warm temperatures in the 1940s 1930s—see those negative coefficients? Then, later on, it applies a positive multiplier so you get a nice dramatic hockey stick at the end of the century.
All you apologists weakly protesting that this is research business as usual and there are plausible explanations for everything in the emails? Sackcloth and ashes time for you. This isn’t just a smoking gun, it’s a siege cannon with the barrel still hot.
It would be difficult to conceive of anything more probative of the value of open government than the Internet-based reaction to the revelations, as dozens if not hundreds of people have self-selected for interest and expertise and started interacting, dividing up the review work, and flowing around the attempted blockage on communication of the dark MSM (“Move along; nothing to see here, assuming there were actually a here here, which we choose not to notice”).
In one area, openness is a big deal indeed—the area called crowdsourcing, or, more prosaically, the decreased transaction costs of community interaction.
The effects of this must be healthy. One scientist reacting to the scandal said: “What keeps scientists honest is not that scientists are more honest than other people—we aren’t—but that we know our colleagues are looking over our shoulders . . . . Scientific conspiracies like the global warming conspiracy are actually quite common. They occur whenever it is difficult for outsiders to check the claims and whenever a pet theory is involved.”
A little-understood dimension of the climate change story is how small is the number of scientists who occupy its core—the supposed “thousands” distill to a few dozen, at most. When the UN’s Intergovernmental Panel on Climate Change (IPCC) trumpets the number of involved scientists, it pads its numbers by including many who have little to do with the product, lack expertise, or even actively object to the conclusions.
This padding leads the public to believe erroneously that many scientists are looking over the shoulders of those producing the work, which in turn allowed the true believers to grab the narrative and create a bubble effect, gaining support from corporate rent-seekers, government power grabbers, academic grant gluttons, anti-energy zealots, and the press.
One cannot say for certain that the revelations puncture the bubble. Maybe the climate change supporters will produce tons of support that does not relate to the tainted work, but this appears doubtful. The lab was crucial to producing a lot of the basic work on which climate change theory rests, and so far the “never mind” defense is not working.
If external motivation exists and participants need not worry about getting paid for their efforts, then opening the gates of communication, enabling self-selection and self-organization, and stepping back to watch evolution in action can produce spectacular results.
East Anglia was not the only research center to hold its cards close to the chest. Many questions have been raised about U.S. data, and at this point the “trust us” gambit is unlikely to work. The government has little option except to make all relevant information available and let the crowd chomp on it. If it turns out that East Anglia was a bad egg, and that solid support for climate change theory exists in other places, then we deniers will eat crow.
But if the government does not react with real and open data, then the hangers-on must draw appropriate conclusions and start defecting.
Even if the climate change advocates were able to tamp things down for a time, science cannot avoid scrutiny when the policies begin to have real bite on energy availability. Unless advocates of drastic action come up with support for their theories that does not relate to the East Anglia data—and at this point there is no indication that such support exists—the public reaction will be outrage. The longer the government waits to reveal this, and the higher the price ultimately paid by the public, the greater the outrage.
If the bubble is over, then all these allies and hangers-on must assess the price of waiting too long to bail. The subsidy seekers might decide to cut their losses and avoid being hopelessly discredited in future fights. Scientists who have supported their fellows because it is impolite to challenge a fellow in his or her area of expertise must weigh the price of solidarity in terms of the damage done to science as an institution. Government people must judge their potential loss of legitimacy when they persist in misrepresenting the strength of the scientific case. Anyone invested in the bubble can see this looming dynamic, so as a matter of elementary gamesmanship the time to desert is now, before everyone else catches on.
Either the government will be forced to come up with better proof of the climate change hypothesis to prevent this, or a misguided public policy will collapse. Either way, it is a victory for the cause of open government, though perhaps not quite what the administration had in mind. I wonder if OSTP will put a link on the Open Government Innovations Gallery.
James V. DeLong is vice president and senior analyst of the Convergence Law Institute, LLC, and special counsel in the Washington, D.C. office of Kamlet Reichert, LLP.
FURTHER READING: Among his other articles for THE AMERICAN, DeLong wrote “Rhett Butler Comes to Washington,” arguing that it would be wise for businesses to band together to defend free-market culture and make their money from our civilization’s rise. He also contributed “Preparing the Obituary” on the decline of the newspaper industry and “Avoiding a Tech Train Wreck” on the intersection of technology and politics. His other pieces include “The Coming of the Fourth American Republic” on how the Special Interest State that has shaped American life for 70 years is dying and “Behind the Green Dam” on how the controversies over Internet filtering are only just beginning.
The American Enterprise Institute's Kenneth Green offers a more sane and flexible policy on climate change in “Climate Change: The Resilience Option.” Green also testified before the Senate in June on scientific integrity and transparency reforms.
Image by Darren Wamboldt/Bergman Group.