Where do you get the idea you don't have that right?

It'll come to you when the guy coming out of the window--who may be the owner of the house--perceives YOU as the threat and shoots your ass.