That's right, humping heavy plywood up or down a ladder is a recipe for a back injury. It's also something you do for a few days and then stop, unless you're a construction worker.
In other words, not a good way to get in better shape. It's like going to the gym and doing heavy aquats and deadlifts for a few days and then never doing them again.
