Barak,
You say, "If God gave me my rights, then only God can take them away. People can threaten me with harm if I exercise my rights, but they're mine nonetheless; I can't lose them to a person or a group of people."
Go ahead and walk into the Post Office/ Federal Building brandishing a loaded .45 1911 and You'll see how fast, your God given rights are removed, and for a perfectly good reason too.