In mid to late season 90% of the time my outerwear is gore-tex lined to provide some wind and rain/snow protection. I also carry a down vest, a neck gaiter and some heavier gloves in my pack. When I'm sitting I'll put the vest and neck gaiter on once I start to cool down.

Affordable Sportfishing Charters and Cruises out of Noank CT - https://www.rowdygirlcharters.com/