When Larry Diamond left for Baghdad in January as an adviser to the U.S. occupation authority, he took all the equipment he believed he needed to help construct a hopeful new nation out of the ashes of dictatorship: the academic models he had crafted over the years as an authority on building democracies, and confidence those models would work.

But the jarring reality of Iraq, with its escalating violence and collapsing civic order, forced Diamond to look for a few new tools beyond those listed in the textbooks. When he speaks now of the models for building democratic countries, he stresses a different set of equipment, which he found in short supply: body armor, armor-plated cars, a huge military presence.

The story of Iraq, this onetime optimist believes, is a tale of missed opportunities.

"We just bungled this so badly," said Diamond, a 52-year-old senior fellow at Stanford University's Hoover Institution. "We just weren't honest with ourselves or with the American people about what was going to be needed to secure the country."

Diamond was a senior adviser to the Coalition Provisional Authority and spent several initially hopeful months in Iraq -- lecturing on democracy, even in mosques, encouraging people to participate and helping shape laws that embodied his vision. He returned to Palo Alto in early April for a short break, then ran into an emotional brick wall, he said, when he contemplated the mess he had left behind.

Last Thursday, when it came time for Diamond to return, he did not get on the plane.

Instead, he was in his office at the Hoover Tower, disillusioned over the desperate turn of events he had witnessed and what he feels was a country allowed to spin out of control, in large part, he says, because of the Bush administration's unwillingness to commit a big enough force to protect Iraqis from militias and insurgents.

"You can't develop democracy without security," he said. "In Iraq, it's really a security nightmare that did not have to be. If you don't get that right, nothing else is possible. Everything else is connected to that."

Few people would seem better prepared for the job in Iraq than Diamond. He is coordinator of the Democracy Program at Stanford's Center for Democracy, Development and the Rule of Law, and he has been co-editor since 1990 of the Journal of Democracy. He has done extensive fieldwork in Taiwan and Nigeria.

He said he had initially opposed the war in Iraq because he felt the United States needed broader international support before attacking, but after the main ground fighting ended last April, he was ready to help.

"Once the war was over, I felt we had a moral and political obligation to the Iraqis to try and help build something better," he said. "That was clear in my mind. I didn't agonize over that. I really had something to contribute."

So late last year, after the Bush administration and the provisional authority outlined their plans for writing an interim constitution and handing over sovereignty on June 30, Diamond said he began to speak with officials about playing a role and implementing some of the ideas he had spent his career developing.

Arriving in Baghdad in early January, he said, he was sober-minded about the challenges but encouraged by much of what he found.

"When I got there on the ground, I was actually hopeful as I met some of the young people, women, civic groups, and their eagerness for change," he said.

"It was mind-blowing, really,'' he added. "There were people who wanted to know how to make democracy work. There were so many positive signs. Civil society was very weak, as you'd expect, but it was beginning to reconstitute itself. There was a lot of energy, a lot of passion, a lot of creativity and a lot of desire to learn. I even had a good experience with some mullahs who supported us."

Diamond said that he had some successes. He said he sought to provide female representatives a guaranteed number of seats in the provisional parliament and helped secure for them a 25 percent stake.

He helped strengthen some of the provisions in the interim constitution supporting the development of civic groups to organize people at a grassroots level, and worked to make the new government structure somewhat decentralized as a way of giving minority groups more of a voice and providing opportunities for grassroots participation. And he instructed, while learning.

In January, he outlined the four basic principles of democracy in a speech at Hilla University, discussing such issues as checks and balances and the rule of law. In February, at a conference in Baghdad on decentralization, he presented a 12-point description of how civil society helps build a stronger democracy.

In another address to Iraqis in late March, Diamond called the transitional law, as the interim constitution is called, the right path to "a true democracy," praised the spirit of compromise he found and promised the Iraqis that their nascent democracy would lead the Arab world.

But Diamond said it was around that time that the insurgency grew bolder, that more Americans and Iraqis began to die and that security appeared to be collapsing. He said he shuddered as he began to see other advisers getting killed on the same roads he traveled.

And then he had what he describes as a painful, transforming experience.

"I had one of those moments when you cut through all the bull," he said. "I was speaking to this women's group, and one woman got up and asked, 'If we do all these things, who's going to protect us?' " Diamond recalled. "That was the moment when I said to myself, 'Oh my God, some of these women are going to be assassinated because they are here listening to me.' It just struck me between the eyes."

As the violence spread, Diamond said, he felt ever more painfully the mistake the United States had made by not sending in more troops to keep the insurgents at bay.

The American policies basically encouraged Iraqis to stand up -- only to face the threat of being mowed down for doing so, he said.

"It was totally hypocritical of us to do one and not the other," Diamond said of the lack of security.

As a result, he said, democratization suffered potentially fatal setbacks. He was angry, he added, not just because optimistic Iraqis were being killed, but because the downward spiral was preventable.

His recommendations for rescuing the situation run counter to some of the policies that the Bush administration insists it will not alter. Diamond said that, in his view, the United States must more than double its current military force of about 135,000 and confront the violent Iraqi militias consistently, while offering political benefits to those who lay down their arms and accept democratic institutions.

The best he can say about the prospects in Iraq now is that, as he puts it, "civil war is not inevitable."

Diamond said that, realistically, he never expected a flawless democracy to emerge in just months. It was more likely, he said, that the legacies of traditional Arab society and dictatorship would have produced some rigged elections, corruption and sporadic violence. But with greater security, there would have been, at the least, a constitution and a more flexible and responsive government.

None of that is likely to happen now, he said, without significantly more American troops and a more assertive military stance.

"The literature stresses the overwhelming need to get the security under control," Diamond said. "Nothing that happened could not have been anticipated. I don't think we were applying the lessons of the past as systematically as they should have been, to put it as politely as possible."

In the past ten days, the US-led Coalition effort to rebuild Iraq as a stable, unified, and democratic state has fallen into crisis. The most alarming aspect is not the Baathist-inspired violence in Fallujah, bloody and horrific though that fighting has been. This has been a limited uprising from the minority Sunni section of the country, many of whose politicians have now entered the peaceful political game. It does not threaten the overall viability of the political transition program in Iraq.

The Shiite uprising that began a few days ago is another story, however. Scholars and historians of Iraq have long warned that an uprising among the Shia would spell doom for the Coalition, and for any hope of peaceful transition to a decent form of governance. We are not yet facing a generalized Shiite resistance. Rather, we are locked in a confrontation with a ruthless young thug, Muqtada al-Sadr, who leads an Iranian-backed, fascist political movement that spouts a shallow mix of Islamist and nationalist slogans in a bid to conquer power.

Among most Shia -- including, crucially, Iraq's most widely revered religious leader, Ayatollah Ali al-Sistani-Muqtada Sadr is a reviled figure, a crude street tough with no religious qualifications and no positive political program, who has used coercion and intimidation as a substitute for genuine religious knowledge and authority. Unfortunately, however, since the Coalition began a crackdown ten days ago on his malicious operation, Sadr has maneuvered brilliantly to portray himself as the leader of a broader nationalist and Islamist insurgency. Now, growing numbers of frustrated and marginal young Iraqi men -- including it appears, some Sunni elements -- are rallying to his cause.

If we do not confront this new resistance in a politically agile and militarily forceful and adept manner, everything we have done to help Iraqis rebuild their country as a democracy could unravel in a matter of weeks.

The democratic transition is moving forward, in many inspiring ways. With US assistance, civil society is organizing, political parties are beginning to mobilize, and hundreds of "democracy dialogues" are discussing the country's constitutional structure and future. Two UN teams are consulting with Iraqis on how to structure the interim government that will assume power on June 30, and how to structure and administer the elections for a transitional government, due by this December or January.

However, elections can only go forward and the transition succeed if the agents and means of violence are brought under control.

Underlying the current upsurge in violence has been the mounting problem posed by heavily armed militias in the Shiite south. Loyal to political parties and religious militants, riven by factional divides, determined to impose an Iranian-style theocratic dictatorship, and lavishly armed, funded, and encouraged by various power factions in Iran, these radical Islamist militias (as well as Sunni and Kurdish peshmerga militias in the north) have been casting a long shadow over the political process in Iraq. In many provinces, the militia fighters outnumber and certainly outgun the new Iraqi armed forces.

Several Islamic fundamentalist parties have been playing a clever double game. As their representatives in Baghdad negotiate and compromise with other parties, exhibiting sweet reason and moderation, their militias have been stocking heavy arms, menacing opponents, and preparing for the coming war in Iraq.

Unless the militias are demobilized and disarmed, a transition to democracy in Iraq will become impossible. Rather, at every step of the way -- from the formation of parties, to the registration of voters, to the election campaign, to the casting and counting of votes -- the democratic process will be desecrated by violence, fear, and fraud.

Key officials within the Coalition Provisional Authority (CPA) have begun to recognize the urgency of this issue. Over the last three months, a plan has quietly been prepared and negotiated for the comprehensive disarmament, demobilization, and reintegration of all the major militias. But this plan, which relies heavily on financial and employment incentives for voluntary compliance by the major militias, can only work if all the militias are disarmed. Those forces that will not negotiate and cooperate must be confronted and disarmed by force.

This brings us to the events of the last week, to the person of Muqtada Sadr, and to the biggest, most ruthless militia that stands indefatigably outside any process of negotiation and voluntary disarmament. A fiery thirty-one-year-old mullah, whose father and brothers were martyred in the Shiite resistance to Saddam, Muqtada has nothing of the Islamic learning and sophistication that would put him anywhere close to the religious stature and authority of an Ayatollah. But he knows how to organize, mobilize, and intimidatehas used the reputation of his father among the poor urban masses, and the language of historic resistance to external impositions, to mobilize a growing following among downtrodden young urban men in particular. His support is confined to a small minority among the Shia of Iraq, but it is the kind of minority, demographically, that makes revolutions and seizes power, and its devotion to his declarations and obedience to his commands is apparently intense.

In recent months, Sadr's militia -- the al-Mahdi Army -- and his loose political movement that surrounds it have been growing alarmingly in size, muscle, and daring. They have seized public buildings, beaten up university professors and deans, taken over classrooms and departments, forced women to wear the hijab, set up illegal sharia courts, imposed their own brutal penalties, and generally made themselves a law onto themselves. As with the Nazis or any other totalitarian movement, all of this street action and thuggery is meant to intimidate and cow opponents, to create the sense of an unstoppable force, and to strike absolute fear into the hearts of people who would be so na?ve as to think they could shape public policy and power relations by peaceful, democratic means.

As with the Nazis, Muqtada has been guilty of brazen crimes well before his effort to seize power openly. A year ago, Sadr's organization stabbed to death a leading moderate Shiite cleric, Ayatollah Abdel-Majdid al-Khoei, who would have been a force for peaceful democratic change and a dangerous rival to Sadr. The murder took place in the Imam Ali mosque, Shiite Islam's holiest shrine. That is the level of respect that Sadr manifests for his own religion. Just three weeks ago, on the night of March 12, apparently in alliance with fighters from other Shiite militias and with the local Diwaniyya police force, the Mahdi Army invaded the Gypsy town of Qawliyya after a dispute over what Sadr's forces alleged were morals violations by the town. After pumping round upon round of automatic rifle fire, mortars, and RPGs into Qawliyya, the Mahdi Army brought in bulldozers and literally leveled a town of some thousand people. We still do not know how many people died in this blatant act of ethnic cleansing (as the towns folk had been warned in advance of the impending doom, and many if not most were able to flee). But at the very least, Iraq now has hundreds of internally displaced people from this incident of terror, and eighteen refugees apprehended by Sadr's forces endured ten days of brutal beatings in the organization's detention center. By the logic of Muqtada Sadr, this is the kind of "rule of law" Iraq needs.

In recent weeks, Sadr's propaganda, both in his oral statements and through his weekly newspaper, the Hawza, have become increasingly incendiary, propagating the most outrageous and explosive lies (for example, that the US was responsible for recent deadly bombings) deliberately designed to provoke popular violence. Finally, on March 28, after months of costly delay, the Coalition finally began to move against this monster. Ambassador L. Paul Bremer ordered the closure of the newspaper, and Muqtada Sadr reacted by ordering his followers to rise up violently against the Coalition. Perhaps in response, the Coalition finally ordered the arrest on April 4 of a senior Sadr aide, Mustafa al-Yacoubi, and 24 others -- including Sadr himself -- for the murder of al-Khoei. About half the suspects, including Sadr, are still at large.

Sadr responded to these arrests by unleashing a revolutionary campaign to seize power. Having already stormed numerous public building in recent months, his followers took over the offices of the Governor of Basra and assaulted police stations in several cities, including Karbala with its sacred Shiite religious shrines to the Imam Abbas and the Imam Husayn. In Najaf his followers invaded Shia Islam's holiest center, the Shrine of the Imam Ali. These attempted power grabs are not new. Last October, Coalition forces intercepted 30 busloads of a thousand heavily armed Sadr followers as they were headed down from Baghdad to Karbala to seize control of its shrines and the central city.

On Monday, the Administrator of the Coalition Provisional Authority in Baghdad, L. Paul Bremer III, declared Muqtada Sadr an "outlaw." Now there is no turning back. If any kind of decent, democratic, and peaceful political order is to be possible in Iraq, the Coalition will need to arrest Muqtada Sadr, crush his attempt to seize power by force, and dismantle his Mahdi army.

We are now embarked on a dangerous and bloody campaign in which, tragically, many more American, other Coalition, and Iraqi lives will be lost. But if we do not confront this military challenge now, while we work to rebuild a broader consensus among Iraqi political forces on the rules of the game and the shape of the new political system, we will lose the second war for Iraq, with frightening implications not only for the peace and stability of that country and the wider region, but for our own national security.