Gordon Brown yesterday took responsibility for Labour's poll debacle last Thursday, and admitted there were no excuses for what he called "a referendum on Labour".

Facing tough TV interviews, he acknowledged he is now fighting as the underdog to David Cameron's Tories.

But the Prime Minister said he felt the hurt people feel at the supermarket checkout and petrol forecourt.

To those who see him as out of touch, he stressed his "ordinary" upbringing, knowing hardship, and the friends he still keeps from Kirkcaldy schooldays represent today's Britain.

He has spent too long at his desk on the detail of trying to solve problems, he said. He admitted making mistakes in abolishing the 10p tax band and presentation of policy. And he pledged to get out more to listen first-hand.

This was the contrite Labour leader yesterday, reconnecting with "families and their finances" and stressing "value for money", but knowing that last week's election results across England and Wales were only the start of his problems.

He faces talk of a leadership challenge, Commons defeat on detention of terror suspects, a fuel tax U-turn is likely, there is revolt once more on the income tax muddle, and a tricky by-election in two weeks.

He has just lost his choice as party general secretary before even taking up the job, Alex Salmond is rampant in his native Scotland and Boris Johnson is the new mayor in London, trialling Tory populism on his front doorstep.

The revolt on the 10p tax rate runs deepest, as it was the dominant theme of last week's drubbing. Chief Labour rebel Frank Field, a former welfare minister, warned yesterday that changes so far promised do not go far enough.

He wrote an article saying Mr Brown had left him "thoroughly confused", Chancellor Alistair Darling's letter of reassurance was "clear as mud" and subsequent ministerial attempts to clarify the compensation package have done anything but.

Mr Brown admitted it is "hard to find" some of the low-paid people who do not qualify for tax credits and who should therefore be compensated. It was not 5.3 million people affected, he claimed, and then struggled to explain how many millions it is. So it is not just the recipients of tax credits who find them baffling: it sounded like their chief architect does as well.

With the Prime Minister weakened, back benchers can only have gained an appetite for revolt over plans to allow detention of terror suspects for up to 42 days. With Mr Brown facing the prospect of Commons defeat, he was yesterday practising the defence that his opponents are soft on terror.

Then there was his refusal to defend the planned 2p per litre increase in forecourt tax, intended as a green measure from this autumn. He left it near certain there is a U-turn coming on that, but left that to Mr Darling.

Mr Brown said yesterday it was not a priority for him to reshuffle his ministerial team. He knows and they know Thursday's loss was all about him. Justice Secretary Jack Straw accidentally admitted as much at the weekend, saying voters had "punished him".

The Prime Minister is not in a strong enough position to crack the cabinet whip, whereas others around him are in a position to start positioning to replace him. Left-wing back bencher John McDonnell yesterday quashed speculation he would stand as a stalking horse, but was one of those calling for a radical change of approach.

There are housing, education, health and constitutional reforms being drawn up by ministers, which may have limited relevance in Scotland. The Prime Minister hopes to put these into his legislative programme for next year. No longer saving that for November and the Queen's Speech, it is there that he is pinning his hopes to start turning things around before summer.