If you use sheet foam the iso-thermal or "I" foam is about three times more efficient than the standard foam. The cost difference is minimal and will be off set by the energy efficiency savings the first year. Even if you use spray foam or panels each inch of the I foam will add an R-15 value so good to add a layer inside and out, one inch would be enough to cut down any convection from the studs.

Go bigger than you think you need on the cooling unit as the larger unit will have to work less. Get a unit that is also a dehumidifier or you will have problems. The prefab insulated panels with either the metal/vinyl or fiberglass facings look like the way to go. A foam interior would be easy to damage.

Also add a radiant barrier to the roof. This is just a layer of mylar and an air gap but it will add another 10-15% efficiency depending on your climate. Also bump up the R value for the cieling.

If possible insulate under your slab and put a moisture barrier down or sometimes odd things will happen like gas or water intrusion. If you can't insulate those rubber mats will help but one more thing to clean up. There are more thermally efficient concrete mixes with vermiculite or other air trapping material but I am not up on these products, something to look into.

The ICF foam building panels are really easy to build with and eliminate most of the framing. They already have an R value of R-25+ when finished and are almost monolithic. Then add whatever paneling you want inside and out and your done except for the roof. One friend built this way and it is so efficient we have to open the windows to have a fire inside or it gets too hot.

Post photos of the build, please.

Has anyone done a cooler/smoke house or is this a bad idea?

"When you disarm the people, you commence to offend them and show that you distrust them either through cowardice or lack of confidence, and both of these opinions generate hatred." Niccolo Machiavelli