Check out the discussion between 12:30 and 17:00 or so.

If you feel qualified to argue, have at it.

Well there are a couple of nuggets in there. Biggest one, neither one of them want to play in the Benchrest universe and both were pretty candid about that.

That tickles.

Goodness, my head is about to explode!

Very good stuff! Bryan Litz is clearly an engineer. I wonder why % humidity vs. dew point though?

I watched the whole conversation and I appreciate their comments near the end when they talked about, assuming that you're wrong so you're open to learning.

Also, earlier they talked about not over reacting to a larger than usual group, it fits in the bell curve if you've gathered a large enough data set.

Does it make a difference? Does it make a difference to me in what I'm doing?

Thanks for posting!

They covered a lot of ground in one video!!

Litz is a good engineer, and it's hard to quantify voodoo.

The main reason I posted was that particular point in the conversation about the statistics centered around 1 moa rifles.

A lot of handloaders for typical hunting rifles would do well to carefully listen to that segment a dozen times or so.

The good ol empirical rule! Stats 101! Can also be applied to many concepts in life!

"The Bell Curve."

Good stuff. Always enjoy an engineering perspective.

Mr. Cortina's body language was interesting to watch as well.

.

Well, you're right with what I think that you are implying, but I doubt that many will really get the significance of what is being said.

Also, one would have to do a LOT of shooting to show that, if they shot enough, eventually they would have shots that reached the outermost deviation that SD's would show as possible, whether they were working with +/- 1 SD or +/- 3SD.

Many guys will not shoot a given rifle & ammo combination that much in a year............hence they don't believe it, unless they are statistically astute.

I guess pure mathematics aside, which clearly show a given possibility, doesn't necessarily mean that everyone (or situation) will ever display that behavior......................just depends on how many samples you want to collect to use as data points.

MM

I watched/listen to this the other day.
One thing I was wondering, was he loading "custom" ammo for this test or using something like Lake City since he was doing the testing for the government

Hey MM, how many on here do you think would go out and shoot say 50-100 rounds in 5 shot groups = 10-20 groups over a chronograph of the exact same load to get an actual SD that’s valid?
If this would happen more and more, they would see the large to small dispersion of group size as noted, unfortunately most on here will shoot 3-5 shots, state they have an SD of 3 then never shoot over the chronograph again unless they think something went wonky.
Also 1500 rounds of the same load isn’t unheard of albeit in my case only 1/10th or 150 rounds was shot over a chronograph.

But, but, but, I once shot a 1/2MOA group therefore I have a 1/2MOA rifle and I am a 1/2MOA shooter! LOL...

The Central Limit Theorem tells us that we should expect a normally distributed mean group size if we shoot enough groups with the same rifle, ammo, and conditions, but what I found interesting was Bryan’s comment that their testing shows a typical SD of 30%. I’ve worked with rifles/loads that showed more variation from group to group than others, and the CLT tells us that the SD in group size between different loads and rifles should also be normally distributed. I have to think that the SD would be smaller with ammo that is carefully handloaded in a node, and that 30% is likely with match-grade factory ammo.

Listened to the whole podcast - the idea of being less wrong opposed to being more right seemed to be a common thread. I can apply the principle of minimizing the downside instead of maximizing the upside in many areas. Eric, as accomplished as he is, isn’t an engineer and at several points tried to oversimplify the already simple.

The humidity issue depicted was a new concept for me to consider more entirely. The example of 140gr Bergers in a 6.5 Creed with same powder charge exhibiting as much as 200 fps variation sure opened my eyes.

Interesting notes on powder charges & primer combos that create a broader range of acceptable results.

Thanks for sharing, Mathman!

As far as humidity goes. I keep my gun room at 30% at 70 degrees, (44.5 dew point) and sometimes it the middle of the summer with the AC going and all the dehumidifiers going it gets as high as 35% or 47.6 dew point.

When I load, I seat the bullets shortly after the powder is poured. The powder measure is covered/sealed so the powder does not gain or lose moisture. I won’t allow the powder to be exposed to the atmosphere any longer than necessary.

As far as sd and accuracy goes, I use three five-shot groups and average them. (True if I get one flier in one group I’ll reshoot that group. Maybe that’s a mistake??? I’ve always blamed the guy pulling the trigger-me.)

This isn’t perfect, but that’s been my practice. I wonder now if I should keep that flyer group in the average? If two group have flyers, I alter the load to get a more accurate or better load.

Interesting how Brian suggested waiting for another podcast to talk about harmonics…

Everybody be careful about which SD is being considered at a given time. There's velocity SD and group size SD.

And that’s exactly what Litz said you shouldn’t do, but common.

So how many groups to get an accurate group SD? I may be wrong, but for me to get a fairly accurate velocity SD is 30-50 shots. So for group SD I would think to be fairly accurate minimum of 10 groups, preferably twice that. Again most won’t go to those lengths.

Regarding the "humidity issue" you may want to check out Keith Glasscock's "Winning in the Wind" videos on YouTube. He did some testing a while back involving storage of loaded ammo in different humidity environments and had some interesting results. He is an F Class Shooter as well and I believe a Process Engineer by profession.

Statistical validity is pretty much why I discount the Ladder Method.

30+ groups.

I just got around to watching the first 17 minutes of the video. I think I'd have a really fun time working with Bryan on his testing. I've come away with a lot of respect for his methods.

The critically important idea he puts forward is that a 1 MOA rifle will routinely print groups as small as half an inch and as large as an inch and a half with absolutely no change in the rifle, cartridge, shooter, or environmental conditions. That's basic built-in variation. So his next point is that if you shoot a 1" group, and then change your trigger technique and shoot a 1/2" group, you don't know whether it was basic built-in variation or real change in performance. A lot of shooters spend a lot of time chasing random variation. No matter, it's all good practice.

The average of three five-shot groups will let you estimate the long term precision of your rifle within about 25%. To get it much closer than that, you need really big sample sizes.

He does make a subtle statistical error or two, but these are not important to the result. The distribution of group sizes is not normal. It looks like a Normal Distribution that has been pushed to the left, with a long tail to the right. So the 68% rule for plus or minus one standard deviation isn't quite right, but it's also not far off. Also, there is no Central Limit Theorem for any measure of dispersion (range, SD, group size), so collections of data do not tend toward normality.

denton,

Could you elaborate on your reasoning here?

It's just a mathmatical truth.

If you are taking inteval/ratio data such as FPS, peak pressure, millimeters, etc., then Central Limit kicks in and the Distribution of Means will have a strong tendency toward normality. That is very convenient for users of the T Test and ANOVA because you don't usually have to worry much about the normality of the data, and the Standard Error of the Mean converges pretty quickly.

Switch to any measure of dispersion, and it's a different world. There is no tendency toward normality. The Distribution of Means looks just as awful as the raw data, and separating normal random variation from real change takes a lot bigger sample. If you're terminally curious, I could scan a page or two out of a text and post it for you.

So for interval/ratio data, we use T and ANOVA. For SD we use F, Bartlett, or Levene's Test.

Right..I shoot one group for conservation of resources. I've been shooting since I can remember. Anymore and I can tell pretty quick when the rifle handles the load well. 30+ groups is about the funniest thing I've seen all day.

Outstanding. I will take a look.

Your intuition is about right. Starting from a baseline of 3 5-shot groups giving you your long term precision within +/- 25%, going to 12 groups gets you to +/- 12.5%, and 48 groups gets you to +/- 6.25%. Quadruple the number of samples to halve the error. It gets out of hand in a hurry.

As you say, most people aren't going to go to those lengths.

I get it fully but these two guys are pushing the cutting edge forward and data collection is typical of most methods seeking a factually supported process.

Not saying that a group of 1 can’t be nirvana but I will say that it is unlikely.

Lightning does strike…but even Lightning has rules.

I’m with you btw - based on my skills I’m not going to burn 30 groups worth of components seeking a conclusion that I likely would not draw a worthwhile, conclusion from…These two fellows are different than me tho and it’s really cool to learn from them.

It's just a demonstration of the difference between the seat of the pants variations most of us use and more rigorous analysis.

I know what they're saying. Applying statistics to this analysis can be fun. I guess. I have a degree in M.E. plus certification in 6 Sigma, so yes Jordan is correct in requesting a sample group of 30 specimens for analysis. Problem is this works for 30 data points, but not with 30 groups of 5 data points.

I have a 21 pound test platform that measures 1 3/16" dia at the end of the straight 26" barrel. This eliminates most of the variation I'm likely to induce into the system. If I place one hole on top of another at 100yds it's game on. If I see a shot fly off where I know I wasn't holding it's game over.

It's not 30 groups of 5 data points. Each group of 5 generates one data point.

I was also curious about the underlying distribution. Can you elaborate on that?

In terms of the CLT, I don't follow the reasoning. It seems to me that group size can be considered an independent random variable, in itself, with some underlying distribution. Random sampling of group size, regardless of its underlying distribution, should follow a Gaussian distribution as the number of samples tends to infinity. As least that's how it seems to me, but I'd be interested to understand this better if I'm wrong.

Exactly. We're talking about different independent random variables; bullet POI versus group size.

Ever weigh all the bullets in a box of components? Boy I sure hope that press operator down at Sierra bullets was having a good day. How about those benchrest primers? Equal amounts of compound? Neck tension all the same? Making consistent ammo is one thing. Then you have to shoot it, and shooting heats things up. It's a mission of controlling variation

The fundamental issue is that all measures of dispersion are differences between data points. Data points behave as we have come to expect. Differences do not. They do not like to be cornered and made to tell the truth. Skewness and kurtosis are even worse. You need thousands of data to get a good grip on those parameters.

For an article I was doing, I created a 20000 shot simulation. With that, I got a very good estimate of the group size distribution. It looks a lot like the distribution of standard deviations: a normal distribution that has been pushed to the left.

Aside from massive reams of data, the shooter, especially with a sporting rifle is w/o a doubt the weakest link in the system..........doing as above mitigates much of that issue.

Statistics like this can drive a sane person mad if you let it.

Compromises have to be made & each shooter, more or less, has to come to grips with his system & what he is willing to do or accept.

I'm not a target shooter, so my needs are less stringent than what an F Class shooter's might be.

Jack Leuba is a honcho at KAC (Knight's Armament) who supplies a lot of ordnance to SF type people. This is an interesting article with a slightly different tack that some shooters might find interesting.

Let's Talk Accuracy

MM

The golden nugget award right there. Thanks

Out of curiosity, I just coded up a similar simulation using 100,000 shots divided into 5-shot groups. The individual shot POI was modelled using a Gaussian distribution. The group size, when defined as the maximum distance between two shots in a group, looked as you described with a skewness of ~0.41. Interestingly, when group size is defined as the mean distance between pairs of shots in a group, the distribution is more normal with skewness of ~0.28.

That was my point earlier, was this custom loaded ammo where the bullets were sorted by weight, diameter, length and brass sorted, etc or was this something like factory ammo, be it Lake City (since this was for the government) or off the shelf ammo like Remington or Winchester?
All of that plays into the data that the test produces. He did not say how they did it and maybe each round was produced to BR standards. I guess I would like to know a little more about how the test was done just to get a clearer picture.

I understand and agree with statistical deviations in group size with any given load, but I have seen some threads interpreting Bryan’s analysis as meaning load development is futile and they’ll all shoot the same with a big enough sample size. My own experience has shown me that that is not true. You CAN develop a load that averages better than another, but there will always be a range of group sizes with any given load.

John

I can tell you, I’m not.

Wearing barrels out shooting groups is a sport I just don’t understand. I shoot stuff, and a group only tells me where the center of my poa is when I leave the house…

You and me both. A colossal waste of time in my world.

John,

Agreed. Bryan mentioned a typical SD of 30% of the mean group size. The absolute mean and SD will vary with different loads and rifles. His point was definitely not that load development doesn’t matter.

Wow! Someone as curious as I am. Well, maybe more curious, since you took it to 100K, and I only did 20K.

Haha, well going from 20k to 100k was as simple as 3 key strokes, so I'm not sure it says much about the relative curiosity.

I'm seeing some very interesting results, however, and am now running a simulation using 1M shots for both 5-shot groups and 10-shot groups, using both definitions of group dispersion (distance between two furthest shots, and mean distance between all pairs of shots).

While you are at it, you might try running 3, 4 and 7 shot groups. That will let you compare average group sizes for different numbers of shots. I published my results a few years ago, and it would be cool to have the results duplicated.

The stats for group size are nasty. I couldn't see any way to do it except by simulation, as you have done. When the going gets tough, the tough resort to simulation.

That’s funny, true but funny. 👍

Haha, very true.

Where are your results published?

I'm already seeing some interesting results and will comment more later.

Varmint Hunter

Things are simpler if you simply worry about how far each shot is from the center of the group, but that is more complex than anyone is going to do in the field. For groups with 5 shots, group size has practically all the statistical strength of standard deviation.

The assumption that's being made here is that the cross hairs are, or were in the center of the group for each shot. What we're contending with here, as you know, are two systems of variation, or noise. There is the variation of the reticle about the target, and the variation of impacts about the reticle. Strangely we tend to blame both systems on the later while suffering from the inability to address the variation of the former. On the mentioned test platform above an 8x32 power Nikon scope is the sight. At 100 yards I can still see slight movements in the reticle about the target. I don't think it's fair to the ammunition to go ape crazy on gaussian distributions for load worthiness estimation when the launching pad is adding considerable noise. I also don't think 3 standard deviations, or 30% scatter is true for all guns and shooters. For a very accurate BR rifle I doubt the competitors would find that amount of variation acceptable.

Very informative video; thanks for posting Mathman!
One takeaway I found useful is that I may start testing primers earlier in my load development. Also the results of Bryan's testing of annealing wasn't what I would have suspected.

Earth to Jordan Smith: Come baaaaaaack, come baaaaaaaaaack !!!!


MM

Using 1M shots was taking too long for the simulation to run.

Using 100k shots (20k groups of 5 shots), a couple of interesting observations emerged:

- Using 30% standard deviation in individual shot POI, the mean group size is 0.92 MOA and the SD in group size using 5-shot groups is 0.25 MOA. Smallest group is 0.2 MOA, largest is 2.2 MOA. When sampling group size of 5-shot groups, the data distribution looks very close to a normal distribution (as if the CLT applied here *grin*), but with a slight offset to the left (skewness of 0.39) and a slightly prolonged right tail (kurtosis of 0.20). Interestingly, when group size is defined as the mean distance between pairs of shots within the group, and the group size sampled, the data is distributed much more normally, with skewness of only 0.30 and kurtosis of 0.06.

- Using 10-shot groups (10k groups of 10 shots each), the mean group size is 1.14 MOA and SD is 0.23 MOA. Smallest group is 0.40 MOA and the largest is 2.32 MOA. The distribution of group size using 10-shot groups shows a larger shift to the left and longer right tail, with skewness of 0.41 and kurtosis of 0.27. The sampled mean distance between pairs of shots using 10-shot groups follows a normal distribution very closely, with skewness of 0.15 and kurtosis of -0.07 (the longer tail is on the left side now). This implies that the mean distance between shots in a group does seem to follow a Gaussian distribution when sample size is large enough.

When Brian stated that his F Class rifle that he shoots .6 moa with won him number 3 in the world I new I was listening to one of few men who say it how it is! Its funny how many people never count fliers.

The fact is a rifle only shoots as well as it does on its worst day and that's how I have always approached it. The worst day is part of the statistical evidence. A true .6 moa rifle will shoot 6" groups at 1000 yards on a bad day! That's impressive IME. If more people were honest with themselves there abilities would for a certainty be inspired to improve.

That's basically what my takeaway was


Trystan

Forgive me for being slow on the uptake, but by the data distribution do you mean the distribution of the group sizes?

Sorry for the confusion. Yes, exactly.

OK, then if I remember the CLT correctly that's not what it says is converging to normal as the number of samples increases.

Holy schidt, you guys still talking statistics? Go out and shoot and really learn something..

Yeah, it’s referring to the sampling of the underlying shot data regardless of the probability distribution of that data, but curiously the group size samples are behaving similarly.

LOL, I shoot plenty and learn from both theory and practice.

Just for fun, I ran this video through the Planet 4MOA translator I've built on ChatGPT. I won't bore you with the details, but the AI's conclusion:

If a guy like me keeps shooting long enuf, there's going to be a distribution of good and bad groups. With all the minute-of-pie-plate groups I shot while living on Planet 4MOA, I'm going to be shooting bugnuts for the rest of my life. It's all statistical!

After reading through this thread a few times I think I'm getting a better handle on this particular concept. So if I'm understanding this correctly (I suck at statistics so my terminology/understanding is probably off), we can't treat group size measurements as independent variables because of the underlying dispersion that those numbers are based on. Since they're not independent variables, the CLT doesn't apply. Therefore if we try to treat those group size measurements as independent variables, the results are unlikely/less likely to accurately model real world behavior. Am I on the right track with that?

On the second part, if instead of using group size measurements, we use the location of each individual shot, those are independent variables, therefore the CLT applies, and we can assume a normal distribution and analysis. Something like that? If so, would I be correct to assume that if I want to get a more accurate model of how my system actually performs within say 20 shots, it would be more accurate to use mean radius than shooting 4-5shot groups?

I appreciate everyone's input in this thread. It's interesting stuff to think about.

That’s essentially what my simulation shows. The resulting distribution depends on how we define “group size,” though for practical purposes, both definitions (maximum separation between any two shots in a group, and the mean separation between all pairs of shots in the group) result in distributions that very closely approximate a Normal (Gaussian) distribution. In terms of accurately predicting real-world behaviour, the simulation suggests that the CLT is closely approximated. If you want an accurate model, calculate the mean separation between shots (or the mean radius, which is essentially the same but uses a fixed point of reference) for each group, and expect your calculations to approximately follow a Normal distribution.

Gotchya, that makes sense. Earlier in the thread when you mentioned "the mean distance between pairs of shots in a group" I didn't entirely get what that meant. It's the mean separation between all the pairs of shots in the group. I can see how that's similar to mean radius in that they're both accounting for every shot in the group rather than just the two furthest shots. I just downloaded OnTarget to play around with it a little bit. Like I mentioned earlier, I know statistics isn't my strong suit but sometimes digging into a subject with something that's interesting is a good way to learn more about it.

https://grtools.de/doku.php

GRT has an excellent group calculator built in and it’s 100% free. I looked at the OnTarget at one time from the link on accurate shooter, it worked well but I dumped it due to the fact it’s shareware/free to try, not free as represented. May be worth 12 bucks but I despise the intentional misrepresentation.

Yeah, you're off to a great start.

I'll point out a couple more interesting results from the simulations. Each shot (100,000 shots total per simulation) was modelled as landing a certain distance from POA following a Gaussian distribution with SD of 1.34 MOA from center. Using 3-shot groups, the mean group size (largest separation between any two shots in the group) was 0.72 MOA with an SD of 0.27 MOA. Using 10 shots, the mean was 1.14 MOA and the SD 0.22. Using 20-shot groups, mean was 1.34 MOA and SD was 0.20 MOA. The mean separation between pairs of shots in a group was the same for all simulations at 0.53 MOA, but the SD obviously decreases as the number of shots per group increases (0.19 MOA for 3-shot groups and 0.06 for 20-shot groups).

The more shots in your groups, the more closely the resulting group size (when defined as the mean distance between all pairs of shots in the group) follows a Gaussian distribution.

Seems like you're getting the hang of it.

Let me try rephrasing a bit for more clarity:

For interval or ratio data (stuff you can measure with a ruler, meter, etc.), we use the T Test to see if two groups of data really have different means, vs. the difference being easily explained by random variation. The T Test tests a difference. Because of the CLT, the T Test is robust to non-normality as long as you have decent sample sizes.

When you get to measures of dispersion, we use the F Test (or one of its cousins). The F Test tests the ratio of two variances (variance = standard deviation squared, a measure of dispersion). Because the CLT doesn't work here, the F Test is sensitive to non-normality.

Group size is a measure of dispersion. So, again, there is no CLT.

It's possible to simplify things by reducing the problem to one dimension. Think of the target in terms of r and theta, rather than x and y. We really don't care about theta most of the time. We just care about how far the bullet missed. So just do stats on r, and you can take a mean and a standard deviation. Now the stats are better behaved. You can simply say that 95% of shots will fall within plus and minus 2 standard deviations, and that works.

I don't think many folks will do standard deviations in the field. Something simpler is needed.

Group size, mean distance from center, and all the rest all contain the same information, wearing different shirts. There is no need for anything beyond group size and standard deviation. For 5 shots, group size is 90% as good as standard deviation.

So for ranges, you can pull out some exotic tools like ANOMR, or you can just punt and do the simulation. Then you can sort the resulting simulation numbers, and note the upper and lower 2.5% points, and you have your 95% Prediction Interval.

It's been fun.... not many are interested in this esoterica. Hope I have shed a little light on the subject.

Thanks for the link, that'll keep me busy tinkering for awhile

That's interesting and makes sense intuitively. With the 3 shot there's a high degree of variability (high SD) because the group size is much smaller than the "true precision" I guess you could say of that system. The 20 shot is a lot closer to the true group size so it's larger in size but with less variability.

I appreciate the insight and explanation. Even if I'm not actually using those other tests that gives a basic understanding of what can be measured and how they can be used. Sometimes just having some intuition for how thing's work helps to understand what we're seeing at the range.

I was thinking about Bryan's statement on that podcast that group size SD tends to be about 30% of the mean group size, in his experience, so I ran some simulations based on my model. I modelled a rifle/load that averages 0.915 MOA for 5-shot groups and one that averages 0.306 MOA. I used 100,000 shots total, but explored what happens if those 100,000 shots are divided up into 3-,4-,5-,7-,10-,20-,35-, and 50-shot groups. Then I looked at the average group size, and the group size SD, in addition to the kurtosis and skewness of the mean shot separation within a given group (average distance between a pair of shots for every possible combination of shots within a group), for all the groups in each data set. Some interesting trends emerged.

Just using visual inspection, the group size SD seems to have an exponential dependence on the number of shots in each group, asymptotically approaching 11% (where SD is 11% of the group size mean), and the skewness follows a similar trend. Kurtosis doesn't seem to have a strong correlation with the number of shots in each group. Interestingly, these trends seem to hold true regardless of the level of precision of the rifle/load, whether 0.915 or 0.306 MOA on average. So to Bryan's point, it seems that SD is about 25-37% of mean group size for 3-5-shot groups, but drops to 11-20% for 7-shot groups or larger. That gives us a good idea of how much dispersion we can expect from shot to shot, whether we have a 1 MOA rifle/load or something approaching BR standards at 0.3 MOA.

0.915 MOA rifle/load


0.306 MOA rifle/load