Is there any real downside to going to the 308?
No.
The downside is in selling the 6.5 Creed to get the 308 because you'll take a bath on it. The cheapest rifle is one you already own. Plus, if you're really sweating the next ammo shortage, then having two rifles that fire near-identical cartridges doubles your chances of finding ammo, and that's a considerable upside.
Okie John