I've hunted with my scoped rifles in as cold as -40 several times. Never tried messing with the scope adjustments much in that weather, but the scopes worked fine and the rifles killed stuff. Just don't breathe on the glass, as klik says...
You, sir, are tougher than I am. I've hunted at -6 a few times and my scope (Leupold M8 4x) worked fine. If it stopped working at colder temperatures than that, it wouldn't matter because I wouldn't be there.