Expat, since you primarily shoot factory loads, I'd go 308.
Having owned a fair pile of each, I've found the 308 generally a bit more accurate, particularly with factory ammo. It also is loaded hotter than the 30-06, and will generally equal it.
Handloaded, I think the 30-06 handles 180 and 200 grain bullets better than the 308, but even a 180 at 2,630 from a 308 is nothing to sneeze at. Having killed a fair bit of the same game with both, I can't say I've ever seen any difference, and it would likely take several lifetimes to attempt to find any.
In identical rifles, loaded to their full potential, I find the 308 kicks noticeably less.
Reading between the lines, I think your gut is telling you to go 308...
