For over five years the war in Chechnya has occupied a central and neuralgic place in Vladimir Putin's political agenda. In unleashing a renewed military campaign in September 1999-abrogating the cease-fire agreement that had terminated the earlier 1994-1996 war launched by then president Boris Yeltsin-President Putin sought to win American and Western acquiescence in, if not support for, Russia's military campaign by framing the conflict as a war on international terrorism.

However, far from extinguishing the conflict, or confining it within the territory of Chechnya, these policies have contributed to the spread of violence and instability far beyond the borders of the Chechen republic. Instead of pursuing strategies that would address the larger socioeconomic crisis of the predominantly Muslim regions of the Northern Caucasus, marginalize extremists, and win broad support from the population of the region, the brutality of Russian military forces and their local allies in the war in Chechnya and the repressive actions of the security services in neighboring republics have fanned the flames of hostility to Moscow and created conditions for the spread of radical Islamist ideologies and the recruitment of new adherents across the Northern Caucasus.

President Putin has treated the problems facing Russia as a product of state "weakness" and has called for strengthening Russia's unity and state power in response. Ostensibly in order to better combat terrorism, he has introduced a series of measures aimed at strengthening Russia's political unity and executive power at the expense of political pluralism, freedom of information, and civil society development. But by weakening or undermining Russia's fragile and weakly developed system of institutional checks and balances on central power, and reducing the transparency and accountability of official behavior, these policies may well be exacerbating rather than mitigating the challenges facing Russia today.

What began as a secular conflict over the political status of Chechnya has progressively been transformed into a wider struggle involving more radical fighters from other Muslim republics with an avowedly Islamist agenda that now threatens to destabilize the broader region of the Northern Caucasus. The past few years have also seen a rising tide of terrorist actions directed against local authorities and security services in other republics of the Northern Caucasus as well as against the Russian government and population more broadly, including terrorist acts aimed at targets in the city of Moscow itself.

From the dramatic seizure of some 800 hostages in a Moscow theater in October

2002, in which 129 hostages died from the effects of a lethal gas used by Russian security services in a bungled rescue operation, to the September 2004 horrific siege of an elementary school in Beslan, Southern Ossetia, in which over 300 civilians died-over half of them children-these episodes have not only challenged the official assertions that the war could be confined to Chechnya alone but have dramatized the inability of the Russian government to adequately protect the security of its population.

The inept and chaotic handling of many of these terrorist attacks has brought into stark relief the poor performance of the security services, the incompetence of local officials, serious intelligence failures, and above all widespread official corruption. In the Beslan episode, to take just one example, the siege was carried out by some thirty-two terrorists, of several different nationalities, who were apparently able to bribe their way across a series of checkpoints to enter the republic and to utilize weapons and explosives stored on the site beforehand. The local authorities and the federal security services proved incapable of coordinating their actions to control the situation, and the Moscow-appointed president of the republic proved completely inept. Indeed, the most courageous and effective actor was Ruslan Aushev, the former president of Ingushetia, a figure removed from power by Moscow for resisting pressure for more coercive policies.

The Putin government has used these events to justify a series of measures which

are ostensibly intended to more effectively combat terrorism but which appear to

have little relation to the real terrorist threat. First, it has refused to seek a political solution to the conflict in Chechnya and has deliberately sought to undermine possible negotiations or international mediation and to delegitimize potential negotiating partners by demonizing a broad array of Chechen political figures within the country and abroad as "terrorists."

Conflating Chechen resistance with international terrorism, President Putin has explicitly refused to distinguish between more moderate figures and extremists and has exaggerated their ties to international terrorist organizations like Al Qaeda.

Domestically, the Russian government has used security concerns to justify ever greater restrictions on freedom of information, on civil rights, and on the role of nongovernmental organizations, particularly those engaged in the defense of human rights. The military and the organs of law enforcement have been given an ever freer hand, rarely if ever held accountable for their abusive behavior and atrocities against civilians.

Refugee camps in the neighboring republic of Ingushetia were closed and the international non-governmental organizations providing medical care and humanitarian assistance to refugees there were compelled to depart. The mass media have largely lost their independence and editors and journalists have been dismissed or attacked for expressing critical views.

A whole series of measures aimed at further centralization of political power and the strengthening of the executive branch have eroded the already fragile elements of federalism and separation of powers in the Russian political system. The autonomy and political influence of regions and republics has been sharply reduced. Parliament, now dominated by a single pro-presidential party, no longer acts as an independent check on executive power, and liberal political parties and their leaders have been marginalized. Most recently, the popular election of regional governors was abolished in favor of their appointment by Moscow, and a discussion is now under way of bringing even local government under tighter central control by eliminating the election of mayors as well.

Moreover, a high proportion of President Putin's appointees to key positions in

the regions are drawn from the military and security services, selected for their presumed loyalty to the president but often lacking political skills or understanding of local conditions. But the substitution of appointed for elected officials does not necessarily guarantee either loyalty or competence.

In the absence of a competitive party system in which political parties help create a web of ties between the central government and local populations, Putin's centralizing measures could well widen the chasm between state and society.

This growing emphasis on centralization, unity, repression, and secrecy is arguably exacerbating rather than mitigating the problems and making state power even more dysfunctional. In Chechnya and in the broader Caucasus region the brutality as well as the corruption of Russian military and security forces and their local allies-and their extensive reliance on torture, mass roundups, indiscriminate executions, disappearances of civilians, and simple extortion-has embittered many toward Moscow and made it increasingly difficult to win "hearts and minds" and build popular support. Indeed, the lack of transparency, and the difficulty of holding Russian officials accountable for abusive behavior, has led unprecedented numbers of Russian citizens frustrated by the unresponsiveness of their own government to seek redress at the European Court of Human Rights in Strasbourg.

Lacking a positive agenda for ameliorating socioeconomic conditions in the Northern Caucasus, the expanding operations of security forces across the Northern Caucasus, the closure of mosques, and the wave of often indiscriminate arrests have served to drive Islam underground and facilitated the spread of extremist ideologies. Without a coherent and sustained program of economic development that would create employment, housing, and education and offer alternative opportunities to an impoverished and alienated population, particularly young males, and absent a serious effort to eliminate corruption, these trends are likely to worsen.

Russia under Putin is facing a somber future.

From the September/October 2004 issue of Foreign Affairs.

As the war on terrorism continues, statistics on terrorist attacks are becoming as important as the unemployment rate or the GDP. Yet the terrorism reports produced by the U.S. government do not have nearly as much credibility as its economic statistics, because there are no safeguards to ensure that the data are as accurate as possible and free from political manipulation. The flap over the error-ridden 2003 Patterns of Global Terrorism report, which Secretary of State Colin Powell called "a big mistake" and which had to be corrected and re-released, recently brought these issues to the fore. But they still have not been adequately addressed.

Now-common practices used to collect and disseminate vital economic statistics could offer the State Department valuable guidance. Not long ago, economic statistics were also subject to manipulation. In 1971, President Richard Nixon attempted to spin unemployment data released by the Bureau of Labor Statistics (BLS) and transferred officials who defied him. This meddling prompted the establishment of a series of safeguards for collecting and disseminating economic statistics. Since 1971, the Joint Economic Committee of Congress has held regular hearings at which the commissioner of the BLS discusses the unemployment report. More important, in the 1980s, the Office of Management and Budget issued a directive that permits a statistical agency's staff to "provide technical explanations of the data" in the first hour after principal economic indicators are released and forbids "employees of the Executive Branch" from commenting publicly on the data during that time.

The State Department should adopt similar protections in the preparation and dissemination of its reports. In addition to the global terrorism report, the State Department is required by Congress to report annually on international bribery, human rights practices, narcotics control, and religious freedom. Gathering and reporting data for congressional oversight is presently a low-level function at the State Department. The department rarely relies on high-quality, objective data or on modern diagnostic tests to distinguish meaningful trends from chance associations. Adopting safeguards against bias, both statistical and political, would enable Congress to better perform its constitutional role as the White House's overseer and allow the American public to assess the government's foreign policy achievements.

A PATTERN OF ERRORS

Congress requires that the State Department provide each year "a full and complete report" that includes "detailed assessments with respect to each foreign country ... in which acts of international terrorism occurred which were, in the opinion of the Secretary, of major significance." The global terrorism reports are intended to satisfy this requirement, but, over time, they have become glossy advertisements of Washington's achievements in combating terrorism, aimed as much at the public and the press as at congressional overseers.

The 2003 global terrorism report was launched at a celebratory news conference in April. Deputy Secretary of State Richard Armitage and Ambassador J. Cofer Black, the State Department coordinator for counterterrorism, outlined some remaining challenges, but principally they announced the Bush administration's success in turning the terrorist tide. Black called the report "good news," and Armitage introduced it by saying, "You will find in these pages clear evidence that we are prevailing in the fight." The document's first paragraph claimed that worldwide terrorism dropped by 45 percent between 2001 and 2003 and that the number of acts committed last year "represents the lowest annual total of international terrorist attacks since 1969." The report was transmitted to Congress with a cover letter that interpreted the data as "an indication of the great progress that has been made in fighting terrorism" after the horrific events of September 11.

But we immediately spotted errors in the report and evidence contradicting the administration's claims. For example, the chronology in Appendix A, which lists each significant terrorist incident occurring in the year, stopped on November 11-an unusual end to the calendar year. Clearly, this was a mistake, as four terrorist attacks occurred in Turkey between November 12 and the end of 2003. Yet it was impossible to tell whether the post- November 11 incidents were inadvertently dropped off the chronology and included in figures in the body of the report or completely overlooked.

More important, even with the incomplete data, the number of significant incidents listed in the chronology was very high. It tallied a total of 169 significant events for 2003 alone, the highest annual count in 20 years; the annual average over the previous five years was 131. How could the number of significant attacks be at a record high, when the State Department was claiming the lowest total number of attacks since 1969? The answer is that the implied number of "nonsignificant" attacks has declined sharply in recent years. But because nonsignificant events were not listed in the chronology, the drop could not be verified. And if, by definition, they were not significant, it is unclear why their decrease should merit attention.

On June 10, after a critical op-ed we wrote in The Washington Post, a follow-up letter to Powell from Representative Henry Waxman (D-Calif.), and a call for review from the Congressional Research Service, the State Department acknowledged errors in the report. "We did not check and verify the data sufficiently," spokesman Richard Boucher said. "... [T]he figures for the number of attacks and casualties will be up sharply from what was published."

At first, Waxman accused the administration of manipulating the data to "serve the Administration's political interests." Powell denied the allegation, insisting that "there's nothing political about it. It was a data collection and reporting error." Although there is no reason to doubt Powell's explanation, if the errors had gone in the opposite direction-making the rise in terrorism on President George W. Bush's watch look even greater than it has been-it is a safe bet that the administration would have caught them before releasing the report. And such asymmetric vetting is a form of political manipulation.

Critical deficiencies in the way the report was prepared and presented compromised its accuracy and credibility. Chief among these were the opaque procedures used to assemble the report, the inconsistent application of definitions, insufficient review, and the partisan release of the report. These deficiencies resulted in a misleading and unverifiable report that appeared to be tainted by political manipulation.

It is unclear exactly how the report was assembled. The report notes that the U.S. government's Incident Review Panel (IRP) is responsible for determining which terrorist events are significant. It says little, however, about the panel's members: how many there are, whether they are career employees or political appointees, or what affiliations they have. Nor does it describe how they decide whether an event is significant. Do they work by consensus or majority rule? What universe of events do they consider?

The State Department announced a decline in total terrorist attacks, which resulted from a decline in nonsignificant events. But without information about the nonsignificant events, readers were essentially asked to blindly trust the nameless experts who prepared it.

The report's broad definitions, moreover, are sometimes too blunt to help classification. Terrorism is defined as "premeditated, politically motivated violence perpetrated against noncombatant targets by subnational groups or clandestine agents, usually intended to influence an audience." The report specifies that an international terrorist attack is an act committed by substate actors from one nation against citizens or property of another. An incident "is judged significant if it results in loss of life or serious injury to persons, major property damage, and/or is an act or attempted act that could reasonably be expected to create the conditions noted."

But hardly any explanation was provided about how the IRP distinguishes significant from nonsignificant events. When is property damage too minor for an event to be significant? How are nonsignificant events identified? Is the IRP responsible for making these determinations too? Has the source and scope of their information changed over time? The corrected 2003 report, the first to list individual nonsignificant acts, defines as "major" property damage that exceeds $10,000. It does not indicate, however, whether that criterion applied to previous reports.

Admittedly, measuring international terrorism is no easy task. Even scholarly reckonings are not free from subjective judgment, and there are inevitably close calls to be made. The most one can hope for in many cases is consistent application of ambiguous definitions.

Unfortunately, in the global terrorism reports the rules have been applied inconsistently. Many cross-border attacks on civilians in Africa have not been included in the reports, for example, even though similar attacks in other regions have been. The report for 2002, moreover, counts as significant a suicide attack by Chechen shaheeds (Islamist martyrs) against a government building in Moscow that killed 72 people. Yet none of the numerous suicide attacks by the Chechen "black widows" that terrorized Russia and killed scores in 2003 was tallied as an international terrorist attack in the latest report. After one such attack, Russian President Vladimir Putin said, "Today, after a series of recent terrorist attacks, we can say that the bandits active in Chechnya are not just linked with international terrorism, they are an integral part of it." If the State Department considers such attacks domestic, rather than international, it should do so consistently from one year to the next.

Another problem is that the staff that prepared the 2003 global terrorism report did not participate in releasing it; in fact, they have yet to be identified. High-level Bush administration officials presented the report to the media, using it to support White House policies and take credit for the alleged decline in terrorism. Even after the report's flaws were recognized, they continued to spin the figures. When the corrected version was released, Black repeated that "we have made significant progress," despite being pressed to acknowledge that last year the number of significant attacks reached a 20-year high. Given the war on terrorism's central role in the upcoming presidential election, such presentation gives the appearance that the report is being manipulated for political gain.

The State Department has tried to explain the report's flaws using language eerily reminiscent of the Bush administration's justification of the failure to find weapons of mass destruction in Iraq. Spokesman Boucher told reporters that previous claims that the war on terrorism was succeeding had been based "on the facts as we had them at the time [and] the facts that we had were wrong." Even Powell partook in the spinning. On the one hand, he announced that "the [original] narrative is sound and we're not changing any of the narrative." On the other hand, he acknowledged, "We will change the narrative wherever the narrative relates to the data."

To his credit, Powell instructed those responsible for preparing the report to brief Waxman's staff on the procedures they had used and the origins of their mistakes. Based on a summary of the briefing by Waxman's staff, much has come to light. Authority for compiling the list of attacks was shifted from the CIA to the Terrorist Threat Integration Center (TTIC), an organization created in May 2003 to "merge and analyze all threat information in a single location." The TTIC provided information to the IRP, which, it was disclosed, consists of representatives from the CIA, the Defense Intelligence Agency, the National Security Agency, and the State Department's Bureau of Intelligence and Research. A TTIC representative chaired the meetings and could cast a vote to break ties on the classification of an event as significant or nonsignificant.

At least this year, chaos prevailed. The IRP's members changed from meeting to meeting-when they attended the meetings at all. The CIA employee responsible for the database left but was never replaced; in mid-process, an outside contractor who entered data was replaced by another contractor. Because of technical incompetence, the report relied on the wrong cutoff date.

Arithmetic errors were rampant. Larry Johnson, a retired CIA and State Department professional, discovered that the total number of fatalities in the chronology exceeded the number listed in the statistical review in Appendix G. According to Black, the errors resulted from "a combination of things: inattention, personnel shortages and database that is awkward and is antiquated and needs to have very proficient input be made in order for to be sure that the numbers will spill then to the different categories that are being captured [sic]." The debacle is more like an episode of the Keystone Kops than a chapter from Machiavelli, but even that analogy is not very comforting.

SETTING THE RECORD STRAIGHT

Despite the data's limitations, the chronology of significant events in the 2003 global terrorism report yields important information about terrorism's trends, its geographical characteristics, and its magnitude.

Time-series analysis, which seeks to discern trends in given phenomena over time, requires a consistent approach to collecting data. The State Department's terrorism report presents time-series analysis, but by focusing on the total number of attacks it misleadingly combines verifiable data on significant events with nonverifiable data on insignificant ones. And because, as TTIC director John Brennan admitted, "many nonsignificant events occur throughout the world that are not counted in the report," one must also be concerned about consistency in the measurement of the total number of terrorist events. Even if the nonsignificant events were listed (and thus could be verified), trends in significant events are more relevant because they track events that, by definition, are more important. Accurately measuring these trends is a prerequisite for understanding the factors that underlie them and the policies that shape them. In fact, an analysis of the revised report reveals that the number of significant attacks increased from 124 to 175, or by 41 percent, from 2001 to 2003-a significant fact indeed.

The detailed chronology also allows analysts to cumulate terrorist events for each country and cross-classify them according to the country where they occurred and the perpetrators' country of origin. These figures can then be related to the countries' characteristics, yielding information that can help policymakers devise strategies to address terrorism's root causes. Using the global terrorism reports for the years 1997-2002, the authors of this article have previously found that terrorists tend to come from nondemocratic countries, both rich and poor, and generally target nationals from rich, democratic countries.

The State Department has rightly emphasized that the threat of terrorism remains serious, but a close examination of its data helps put the magnitude of the threat in perspective. In 2003, a total of 625 people--including 35 Americans--were killed in international terrorist incidents worldwide. Meanwhile, 43,220 died in automobile accidents in the United States alone, and three million died from AIDS around the world. Comparative figures, particularly when combined with forecasts of future terrorism trends, can help focus debate on the real costs people are willing to bear--in foregone civil liberties and treasure--to reduce the risk posed by terrorism.

CHANGING TRACKS

The State Department currently uses, and Congress accepts, nineteenth-century methods to analyze a twenty-first-century problem. To prevent errors of the type that riddled the 2003 global terrorism report, Congress has two alternatives. It could reassign the State Department's reporting responsibilities to a neutral research agency, such as the GAO (the General Accounting Office, recently renamed the Government Accountability Office) which routinely uses appropriate statistical practices. The problem is that the GAO has little foreign policy expertise and does not necessarily have access to the (sometimes classified) information that goes into the reports. Alternatively, Congress could keep the reports within the State Department's purview but demand that its practices for data collection and analysis be improved and that the reports be insulated from partisan manipulation.

If responsibility remains within the State Department, Congress should establish a statistical bureau in the department to ensure that scientific standards are respected in all reports, thereby elevating the status of data-gathering and statistics there. The bureau would promote consistency, statistical rigor, and transparency. When appropriate, it could seek input from the scientific community. And, while respecting classified sources, it could also insist that sufficient information be released to independent analysts for verification.

To overcome conflicts of interest facing political appointees who issue government reports, the State Department should adopt rules similar to those that govern the production and dissemination of key economic indicators. Career staff who prepare the reports should be given an hour to brief the media on technical aspects of the data, during which time political appointees would be precluded from making public comments. (After the hour elapses, it is expected that political appointees would offer their interpretations.) Career staff should be protected so they can prepare mandated reports without interference from political appointees and then present them for review by the statistics bureau. Once the reports are finalized, but before they are publicly released, they should be circulated to designated political appointees who need to prepare for their release. Disclosure dates should be announced long in advance to prevent opportunistic timing by political appointees.

Last October, in a candid memorandum to top aides that was leaked to the press, Secretary of Defense Donald Rumsfeld admitted, "Today, we lack metrics to know if we are winning or losing the global war on terror. Are we capturing, killing, or deterring and dissuading more terrorists every day than the madrassas [Islamic schools] and the radical clerics are recruiting, training, and deploying against us?" The statement was a stinging acknowledgment that the government lacks both classified and unclassified data to make critical policy decisions. It is also a reminder that only accurate information, presented without political spin, can help the public and decision-makers know where the United States stands in the war on terrorism and how best to fight it.