I'll do the Cap'n obvious part up front.
If your puffy is being compressed at all either by being at all snug itself, or being squished under your outer layer, you're defeating it's purpose. A shell over a puffy needs to be LOOSE or you're losing insulating properties.
Dead-air space is your friend.
Yep. Nothing is tight and puffy is fully puffed.