The 338-284 has the SMALLEST capacity of the 4 cartridges mentioned.
Therefore with all other things being equal (they never are in reality) the 338-06AI would be fastest, then the RCM, then the 338-06, and finally the 338-284.
The AI has nearly 6 grains greater capacity than the 338-284.
Is that 6grs capacity considering combined overall length with bullets seated to magazine constraints? I think when you load both to magazine length with the same bullet you will get more capacity from the 338-284. The shorter case has less capacity displaced by the bullet shank when both are loaded to LA mag length of say 3.3
