Now the posh party for London’s middle class and elite, it has turned its back on its core voters – and the numbers prove it ...