We all love each other. It was no doubt from the beginning," the actor told PEOPLE of his 'Modern Family' castmates ...