Not so long ago a pub in rural England was trashed by a gang of yobs as they indulged in a drunken brawl. Some of them spent the night in cells at Abingdon police station.

This was not just another night of excess in today's alcohol-crazed Britain. The owner of the pub was promptly and courteously offered a lot of cash - in high denomination notes - to pay for the considerable damage. His staff were generously tipped. And he was chided for getting the police involved, because this showed that he "had no sense of humour".

So, these were no ordinary yobs. They were members of the notorious Bullingdon Club, the drinking society that allows fresh-faced toffs to interrupt their no doubt assiduous studies at Oxford University for occasional bouts of bruising boozing and good old-fashioned vandalism.

On Tuesday morning, John Humprhys, the grandly forensic inquisitor-in-chief of Radio Four, conducted what was by his standards a fairly bland interview with David Cameron.

It was only at the end that Humphrys drew blood, when he raised the issue of Cameron's membership of "Bullers" in the 1980s. Cameron was embarrassed, just as he had been earlier when a picture of him and his colleagues posing in their £1000 Bullingdon uniforms, looking disdainful, dandified and decadent, was published in national newspapers.

George Osborne and Boris Johnson were also members of that infamous club in the 1980s. It begins to look as if you need to have been in the Bullingdon if you want to get to the top in the modern Tory party. Should senior Tories' past association with this absurd organisation be held against them?

I think it probably should. It points not only to frivolity, but the abuse of privilege. I know that to write this might seem patronising and that most of us, in our youth, did things we later regret - and don't regard those things as being indicators of what we later became.

But the child is father of the man. And while today's Britain is a dissolving society, and what is and is not socially acceptable is probably changing more rapidly than ever before, the thorny issue of privilege, and its abuse, remains as potent as ever.

I was an undergraduate at Oxford in the late 1960s. I loved the place but did find it something of a hothouse. Occasionally I felt a sheer physical need to get out, to breather fresher air. First, I joined the Ramblers, but found them boring.

Then a friend suggested that I should go beagling with him. He was a whipper-in with the university pack. I demurred because I had an aversion to any kind of hunting.

But he told me that it would get me into the countryside, there was a good hearty drink of mulled wine before each meet, and many of the "followers" wandered around in the distance and never got near a kill.

So I went along a few times. It was just as I had been told. I met countrymen who had never been out of Oxfordshire in their lives; I also met very rich students, who were absurdly well-travelled, jolly and courteous. They also seemed to have an inbuilt inability to understand that for many people, life is something of a struggle.

Most of them were at Christ Church, the poshest of the many Oxford colleges (it is so grand that its chapel is actually a cathedral). Unusually for undergraduates in the 1960s, many of them possessed cars. I suspect they were members of the Bullingdon. In my superficial encounters with them, I felt I should dislike them, yet it was almost impossible to do so.

A few years later, when I was working as a journalist in Edinburgh, the current Prime Minister was the leading student of his day. I witnessed him addressing various very long meetings: not student meetings, but meetings of low-paid university staff, in grim basement rooms. This was what he did in his spare time.

He was not frivolous. He was earnest to a degree (and, like Cameron, he got a very good degree). The Cameron-Brown student dichotomy probably should not be presented as a class collision. But it does undoubtedly reflect two contrasting versions of life, two utterly different and irreconcilable social narratives.

Meanwhile, there remains a curiously potent nostalgia in Britain - Scotland, as well as England - for the heyday of frivolous Oxford.

The great laureate of this vanished world was Evelyn Waugh, who, when he was at Hertford College, became a persistent, obnoxious and rowdy drunkard.

Later he wrote Decline and Fall and Brideshead Revisited, fine novels about that supposedly magical period. Maybe those who try to hang on to that world, like members of Bullingdon Club, are simply suffering from arrested development.