Unless one is doing work that puts them in extra danger smaller gun makes more sense. Smaller gun especially hammer-fired DAO offers more concealment options and can be deployed more quickly.
I have to disagree with your assertion that a small gun deployed more quickly. For me, it's much easier, and hence quicker, to get a proper grip on a full sized frame. As for carry options, I live in Colorado, so most of our weather provides plenty of leeway to dress around a 15 round 9mm frame. For what I need, I find such guns provide a good balance between concealment and capabilities.
Where you live, or for your hand size, that might not be the case, but for me, it's what generally works best.