A lotta times if I'm moving the jacket ends up becoming unneeded weight in my pack.
That's true, however...this is assuming you can keep moving. A sprained ankle in a cold place will require a warm jacket in your pack if you hope to get home. A puffy jacket, either down or synthetic, doesn't add much weight and will compact a lot. it's good insurance.
It will also add one additional round trip when you are packing meat and must choose between a quarter or an unused jacket.
A down jacket will weigh under a lb. That's going to add another trip? You're guaranteed to not slip and sprain an ankle while you have 60lb of meat on your back? You're more apt to get hurt while carrying a load than when not. If it's 20 and blowing, you darn well better have that tiny 10 oz. jacket on top of your pack.
If it is 20 and blowing that did not occur instantly. If you can't read the possibilities of that days conditions, and furthermore ignore the available forecasts, then take it a step further by not keeping a reasonable distance from camp if weather does begin to turn, then by all means carry all the stuff with you that you want each and every day.
A certain amount of awareness of the environment and situation goes a long way toward proper preparation.
Personally, I certainly have those extra layers along and available in a bag containing gear for inclement weather. Some years (not many) I have actually even used it.
