You're under-thinking the sizing, based on an assumption that the chamber is always large enough. It's true that leaving the bullet a larger size than .001-.002" over groove is fine, but if you don't measure, you can certainly end up with larger groove dimensions than you might think. 9mm is a classic example of that. Some 38-55 rifles are another example where just relying on chamber dimensions will leave you with an undersized bullet and wondering why the rifle doesn't shoot well.
Better to understand why/what bullet size does than to just use an arbitrary rule of thumb like fired case dimensions.
