It was once one of the largest private residences in the country. But these days, some lucky students call it home.