You came in late and instead of quietly going to your room, you admittely wandered around and made no effort to be quiet. That is nothing but fucking rude. In fact, you were so loud that the owner no longer could think that it was late night guests arriving but instead some hoodlums possibly robbing him.
He owns the place.
He protects the place.
You should appreciate that or leave.