Well, I have a hard time accepting that survey as being anything even CLOSE to valid. In the vernacular, it's about as useless as teats on a bull.
Self-report surveys are about the worst form of data gathering, statistically speaking, that you can get. People "misremember" according to their biases and prejudices (good and bad), for one thing; for another, the people who respond to self-report surveys tend to be cranky and dissatisfied compared to the general population.
I've had more than a little bit of statistical training, and measurement method/quality management training, and I definitely get what you're saying about lack of controls in definitions & reporting of problems. No question you or I can design tests that are far more objective and repeatable. I'm sure it wouldn't cost us more than a few million dollars to run that test, by the time we bought all the guns, ammo, & time for the people involved
And yes, some guys are going to exaggerate failures, limp wrist, use crappy ammo, have buyer's remorse, etc. And some guys will cut the gun some slack and under-report failures, if it's their fave brand. I think the Ayche und Kaye owners are all like Porsche owners, and will never admit a problem with their guns
Two points though - given enough data, the excessively favorable and excessively pessimistic reports will wash out one another, and useful trends can be spotted. Second, having owned 20+ 1911's, and shot a number more, the averages mirror my experience. In particular, I have owned or shot 6 different Kimbers, including 4 custom shop guns, and 3 of them were not reliable.
Dumping an S&W 1911 (15.9% problems) in favor of a Wilson 1911 (12.5% problems) or Baer (13.2% problems) solely based on the survey would be unwise, given the small differences and the limited amount of data. But when hundreds of reports are in on Colt (17%), Springfield (30%), and Kimber (44%), I believe those trends are informative.
It's also interesting that the Glocks ("which always work") are indeed very good, but 10.4% report problems, which is close to Wilson & Baer's batting average...
