Insulation can make a huge difference. Your bag insulation under you is useless because your weight flattens it out. You're essentially sleeping on cold air. I have an R3 pad that's much warmer than just air. When it gets real cold, a simple ensolite pad on top of that makes it good down to 10 or lower. An R5 or higher is well worth it for cold weather camping.
The problem is that R's add weight. You have to determine how much weight you're willing to add.
