In the wake of the financial crisis the West has witnessed a rise in radical politics of the right and left. One explanation is that the crisis has exacerbated the effects of more longstanding social and economic change.

Two seminal research papers by Nobel Laureate Angus Deaton and Princeton economist Anne Case shed new light on this phenomenon. In a quite remarkable piece of research Deaton and Case examine the effect of adverse economic change, poor education and low social capital on white, working-class Americans.

While other Americans and those in developing countries have seen improvements in life expectancy, mortality rates have risen for white, non-Hispanic middle-class Americans since the late nineties. The rise is especially pronounced among those in middle age with a high-school degree or less.

The differences between racial groups are stark. Mortality rates among black and Hispanic Americans are declining. In 1999, the mortality rate of white non-Hispanics aged 50-54 with only a high-school degree was 30% lower than that among blacks of the same age group. By 2015, it was 30% higher. The authors document similar crossovers between white and black mortality in all age groups from 25-29 to 60-64.

The one, shocking precedent I can think of for such a deterioration in life prospects comes from post-Soviet Russia. Between 1990 and 1994 life expectancy for Russian men dropped from 64 to 58 years and for women from 74 to 71. The causes were complex but are widely thought to include economic and social instability, excessive smoking and alcohol consumption, poor nutrition, depression and a declining health care system. This episode provides part of the explanation for the subsequent rise of Vladimir Putin.

The rise in mortality rates recorded by Deaton and Case is specific to the US. In other comparable advanced economies, mortality rates have been declining. Indeed, in Europe mortality rates are falling for those with low levels of educational attainment and at a faster rate than that for the highly educated.

What is happening in the US?

The obvious explanation is that less-skilled Americans have suffered most from the loss of jobs in industries such as steel, car making and coal mining. The squeeze on the typical earner is well documented. Between 1999 and 2015, median US household incomes fell by 2% even as the US economy expanded by more than a third. It is no secret that the fruits of US growth have not been flowing to lower-skilled, less-educated Americans.

Yet Deaton and Case believe that this is not solely about incomes. If it were, less educated black and Hispanic Americans, whose incomes have also been squeezed, would have suffered rising mortality rates too.

Rather this seems to be a life of 'cumulative disadvantage'. Poor education undermines job prospects and fuels family instability and poor health. A stagnation in wages which has, for many blue collar Americans, been underway since the 1970s, has bred a sense of hopelessness. This contrasts bleakly with the post-war prosperity and optimism in which parents and grandparents came of age. Some Americans have found that the apparently ever upward escalator of progress has gone into reverse. The widening gap between experience and the American dream is reflected in a rise in "deaths of despair" – from suicide, alcohol and drugs, especially opioid painkillers.

Deaton and Case's papers make shocking and gripping reading. Debo, Alex and I have spent quite some time discussing its findings in the last fortnight. Three things stand out to us.

First, high-level economic indicators such as GDP growth or average incomes provide a very partial guide to the reality of life for many people. The distribution of income and of outcomes – from mortality to education to health – matter at least as much.

Second, the winners and the losers in society change over time. For most of the last 30 years policy has tended to focus on the disadvantages that run along lines of income, race and gender. Case and Deaton's analysis paints a more complex picture.

Third, education matters more than ever. The gap in incomes and mortality between more and less educated Americans is increasing. As Professor Case remarks, "It looks like there are two Americas. One for people who went to college and one for those that didn't".

The very complexity could induce a sense of despair about restoring hope to these communities. But our reading of recent history is that government policy can triumph against apparently intractable social problems.

In the 1980s and 1990s, UK social policy focussed on reducing pensioner poverty. That aim has been achieved. Today poverty is far more concentrated among those in low paid work than pensioners.

Changes in Britain's welfare system have also helped raise the UK's employment rate above America's once famously high jobs rate.

In the last 20 years Germany has slashed its stubbornly high unemployment rates through reforms which help and cajole people into work.

The success of London's state schools, where disadvantaged pupils outperform those in the rest of the country after years of lagging behind, has been credited to a range of policies, including the London Challenge and Teach First.

Well designed, properly financed policies can make a big difference to seemingly intractable social and economic problems. For their part Deaton and Case suggest that American politicians could follow Europe by providing more generous social security and more accessible healthcare.

What is clear is that GDP growth is not a tide which lifts all boats. Deaton and Case's ground-breaking work shows that the distribution of income and opportunity is at least as important.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.