I'd rather lose it from the body and it's the cheapest place to lose it from, not to mention being healthy. It is still weight you are carrying up a mountain. It is way more comfortable to carry extra pounds in your gut than to carry it in a pack strapped to your shoulders but you are still carrying it.

Plus, if you lose 2 pounds off your body, it won't make you shoot worse. If you lose two pounds off your rifle, especially if it's a hard-kicker, it will probably make you shoot worse.