FWIW I've found much the same thing with rimmed cases from .22 Hornet to 9.3x74R: they all start out a bit smaller than the chamber, and fireform to fit after the first time they're used. Perhaps the most extreme examples are my .303s, especially the wartime-production No. 4 I have.
I have never worried about it. I work up loads in fireformed cases and neck-size only, to save excessive working of the brass. I'd suggest keeping the brass for each rifle of the same calibre separate though, which is what I do.
