I think it is in the die. When you size, that brass has to go somewhere and if the die is "oversizing" your brass, that could be your issue.
I have a buddy that has the same issue with his 308. He trims his brass .004 more than needed and after sizing, it is on the money for the length he wants.

I may not be smart but I can lift heavy objects

I have a shotgun so I have no need for a 30-06.....