I've had quite a few of both. If you're shooting deer at under 300 yards, probably not much difference. If you're shooting heavier bullets -- 180 grains and up the 06 shines.
Personally, I'd rather shoot the 06 at lower pressure than what the 308 is loaded. The 06 cases last a very long time. I don't care to hotrod reloads. My favorite cartridge could very well be the 06.
Plus, I don't like short action rifles - it limits the seating of the bullets and that influences my preference.
My preference is likely different than others. that's why they are called personal preferences.