Here is my perspective:
I don't generally carry a primary weapon without a holster. The average gun toter never gives it 2 thoughts, but most fights are physical long before bullets go to flyin'
I have had more than a few occasions to end up physically fighting with someone, while armed with a service weapon.
The punchline is that when you are rolling, physically fighting someone, stuff goes flying.
As a weapon retention instructor, I can tell you that this is actually old news.
When you get punched in the face/knocked down, if you have (had) a gun simply tucked in a waist band, odds are more likely than not, it will not still be there.
It is bad enough rolling, bare knuckling it out, but the last thing I want to do is to have to have a brutal ground fight AND be struggling for a loose gun.
I don't think anyone is going to argue that it's more secure, but I don't think it's necessarily that precarious either. Last time I changed the wife's oil I forgot I had the gun in my pants, and I didn't lose it. I wasn't fighting anyone, although from the swearing it might have seemed that way. It's a compromise, like anything else.
I also try to avoid going around and getting punched in the face but YMMV 
.
Most people try to avoid getting hit in the face or anywhere else in a fight, but that is the reality of physical fights.
While you may think carrying w/o a holster is not any more precarious than with one, even when fighting, this leads me to believe that perhaps you have not actually been in a physical fight while trying to retain a loose gun. It has very little in common to changing the oil on a car.
The fact is that you don't get to choose the fight that comes to you, you just get to deal with the fight that is presented to.
That may be getting sucker punched from the side, by thugs playing the "knock-out game", or being jumped by 2-3 dudes who want what you have. That happens far, far more often than standing 10-20 feet apart exchanging shots with another duelist.
This reminds me a lot of guys who insist on not wearing a helmet when they ride a crotch rocket at break neck speeds, because they have "never needed one yet".
