The great waves of civil rights legislation and urban policy since the 1960s had successes and some failures. The great success has been the establishment of a much larger and better educated Black middle class.

Forget the high profile achievements of the few—two of the last three Secretaries of State, the current President, and Attorney General, for example.

It is much more important that millions of African Americans all over the country are getting better educations, better jobs, and better housing than ever before.

But there have been great failures as well. The deterioration of urban life for those who haven’t made it into the middle class is an ongoing tragedy that 50 years of policy have done little to help. There are reasons to believe that 50 more years of the same approaches will also fail.

I’ve written about some of these obstacles. Progressive social policy makes cities expensive and cumbersome places for the kind of businesses that employ poorly educated, low-skilled workers. Many of our well-intentioned urban policies end up steering jobs away from the people who need them most. Municipal labour practices make everything from infrastructure construction to normal urban administration cripplingly expensive, creating an expense structure and a tax burden that basically makes life impossible for many of the small businesses that could offer jobs to low skilled workers.

None of these points has anything to do with race, and if readers will forgive me, I’d like to keep the discussion non-racial for a while.

Don’t worry, the race break won’t take long; we are Americans and one thing that unites us through time and across racial and class lines is a tendency to see every question in racial terms.

From the Moynihan Report on, every discussion of urban issues that looks at factors like illegitimacy and youth violence gets caught up in a racial maelstrom. Some (mostly on the right and usually white) are quick to find fault with “black culture” or seize the opportunity to launch a discussion of IQ scores. From the left and often, though by no means exclusively, from African Americans, comes a chorus alleging the “racism” of those who cruelly want to “blame the victims.”

We should try to come at this from a different angle. As I posted earlier this month, there’s a pattern to urbanization found all over the world among many races and cultures. First generation migrants from the farm still have the social discipline and work attitudes that characterise rural societies. Most young migrants find a foothold in the urban world and move fairly seamlessly from a rural value system to an urban one.

Some, however, through bad luck or bad choices, fail to make the transition. They, and their offspring, form what Karl Marx (who was thinking about Europeans when he coined the term) called a lumpenproletariat: deracinated, alienated, largely incapable of bettering their condition either through labour or through political activity.

Let’s make clear that lumpenproletarian is not a synonym for “African American living in the inner city.” Many urban Blacks, including many poor people, are hardworking Americans raising their children, paying their taxes, supporting religious organisations, and generally going about their lawful and righteous business in exemplary fashion. And many of the people who do fall into Marx’s category aren’t black.

Having made those important clarifications, we need to go on and say that some key urban problems are lumpenproletarian problems: the problems of large groups of people who have become disconnected from the habits and institutions through which their lives can be improved. This population, often marked now as in Marx’s time by alcoholism and other forms of addiction, and then as now enmeshed in a culture of violence and crime and crippled by weak family structures, encounters many difficulties and makes life more challenging for those who live in and around it.

In Marx’s day, the lumpenproletariat was smaller and less-developed than it is in the 21st century. Poor public health, the absence of social safety nets, and the bad quality of food and water available to the poor meant that the urban migration was a fairly Darwinian process; not a lot of failed migrants survived and reproduced. High mortality rates from violence and the consequences of drug abuse and the astronomical abortion rates among this group today show that some of these conditions still exist, but life expectancy for the urban poor remains much higher than in past centuries.

Much of what we see in our cities today is not a uniquely American problem and certainly not a race problem; it is a problem that will become more serious in the U.S. and elsewhere as this century’s mass migration from the country to the city continues all over the world. In much of East Asia, South Asia, Africa, the Middle East, and Latin America, an urban underclass in all the colours of the rainbow is becoming a serious concern.

It’s important for Americans to look at our urban problems through non-racial lenses from time to time; a fresh perspective can free us all up from stale knee-jerk responses and help us think more clearly about what we can—and can’t—do to help. That can lead to another surprising turn; when it comes to the inner cities, some of the most important limits on what government can do come from liberal rather than conservative ideas about government’s proper role.

While some inner city issues are related to policy, others stem from the social and individual dysfunction that characterises communities who have lost the structure and stability of rural life but have not acquired the structure and stability of the modern metropolis. The old skills and strengths disappear; they are not replaced with new ways of earning a living, constructing a family and engaging effectively in an urban social and political environment.

Government and bureaucratic institutions can’t do much to fix these problems. Drug and alcohol addictions, the consequences of abandonment or violence in the home, the corrosion of soul and self esteem that comes with years of unemployment, forced recruitment into gangs, a culture of sexual exploitation and violence marked by contempt for women and homosexuals: these are the kinds of evils Jesus spoke of when he told his disciples that some evils are only cast out by fasting and prayer.

The answer is not religious in every case, but when we are talking about uneducated people who have grown up in wasted social landscapes with little exposure to positive role models, little experience with or preparation for employment in the formal economy, and for whom hard work at low pay may be the only road forward, the only realistic hope is often the power of faith and the support of a strong and focused religious community.

The kind of faith that historically works for people in these situations is not cool, rational, and sophisticated. The Unitarian Church has many merits; helping uneducated, impoverished drug addicts in the inner cities find new meaning and structure in their lives is not one of them.

The faith that works in lumpenproletarianized urban areas, not just in the United States but all over the world, is hot religion: Above all, Pentecostalism, but also the hotter forms of Islam, some intensely emotional forms of Hindu and Buddhist devotion, charismatic Catholicism and other faiths that offer immediate contact with God, black and, white moral teachings that help people regain their bearings and remake their lives, and strong supportive communities who watch over new converts and help neophytes in the faith grow and deepen their faith while welcoming them into a supportive fellowship.

Very often, hot religion features authoritative and even authoritarian leadership: preachers or imams who speak with conviction and provide community leadership as well as spiritual guidelines.

In the past, secular ideologies have sometimes provided the leadership and community that enabled newly arrived urban migrants and their children to build new lives. Nationalism, socialism, and fascism alone or in various combinations have played these roles in the past; in some places they may still be capable of doing it today.

But in general, the last 50 years have seen the decline in the ability of secular causes to mobilize psychological and community energies among the unlearned and the marginalized. Increasingly, it is God or nothing for poor urbanites today.

In some countries, where government and religion are hand-in-glove, government can act to promote religious revivals, religious education, and the presence of religious institutions and orders in poor parts of town. Here that is strictly verboten, and rightly so. The city can and should provide police protection for a Billy Graham-style revival meeting, but it should not rent the tent or order city employees to attend; public school teachers should certainly not require their classes to go.

The secular nature of our government necessarily limits its ability to do “soul care.” When government steps into roles that churches traditionally play—when, for example, “grief counselors” work with schools in the aftermath of a tragic shooting—all the religion has to be carefully and thoroughly purged.

Let me say clearly that I think this is correct. Government and religion should not mix. But the consequence—which many progressives do not fully grasp—is that much of the most important work in the inner city cannot, ever, be accomplished by government programs and government funding.

Let’s look at what may be the core task of rebuilding social cohesion and strength in urban communities in the U.S. and abroad. A young woman who finishes high school and marries before having children is much less likely to be poor than one who drops out and has a child out of wedlock. If she finishes college she is even less likely to be poor—and her marriage is significantly more likely to last. This is true for women of all races; poorly-educated single mums in our society do not have an easy time, and neither do their kids.

Reducing poverty in the inner city means creating a situation in which more young women are both willing and able to make the choices that ensure better lives for themselves and their kids. Of course, this also implies some changes among young men.

For one thing, more of the young men in the inner city must be what in the old days people called “the marrying kind.” They must come to see manhood defined as real maturity—a life fulfilled in service to family and community. This is a radically different (though much richer) masculine ideal than the cartoon-like visions of omnipotence that many adolescents and boys have of manhood. Those visions are especially compelling when boys do not live at close quarters with a real man whether a biological father or not.

Government cannot do much under our system to teach boys what being a man really means. It cannot do much to teach girls to value themselves in a hostile and exploitative environment. It cannot give either sex the inner strength to mature under hostile conditions in a confusing environment filled with mixed signals, intense and exciting temptations, and a full service array of the most enticing and dangerous drugs that the world has ever known.

In a strong and healthy community, this teaching comes from parents and youth leaders and it is reinforced by role models, strong institutions, and clear examples on every side that playing the game “right” pays off reasonably well. In a broken, deracinated third-generation lumpen community, these systems decay from decade to decade.

There is very little, in the U.S. at least, that government can do to change that.

Coming to grips with this is hard for the progressive mind. The substitution of the rational state for irrational and haphazard community institutions is one of the core values of progressive thought. Reducing the private sphere and increasing the public sphere in the name of universal justice, reason, and democratic control is what 20th century progressives saw as a core mission.

But when it comes to this particular case, that model doesn’t work. In another context, Rick Warren once said to me on a visit to Rwanda that if the only way to end poverty in Africa was to employ hundreds of thousands of college trained bureaucrats and experts at six figure salaries—then poverty in Africa would never end. This is as true in our inner cities as it is in the rest of the world. It is not just a matter of money, but of mission and mandate. The inner cities need stronger cures than government is licensed to provide.

To slay the demons that stalk our cities and prey on our youth, we are going to have to call on stronger forces than city hall. That won’t be easy and will require deeper commitment and new kinds of resources and partnerships—and no matter what we do, this is hard and slow work. Lives change one by one; hope grows in one heart at a time.

I’ll return to this subject with some thoughts about what a new stage in the history of urban America might look like.