I don't know if this is a concern or not - but I know that there are legal ramifications to using handloads in a self defense situation when it comes to handguns. Most every CCW class teaches that you should use commercially loaded ammo appropriate for the situation or you could leave yourself WIDE open for a civil lawsuit even if you were otherwise 100% in the right.

As far as using shotgun in selfdefense situations I would think that the same logic applies - i.e., whatever you use, from 7-1/2 target loaded birdshot for appartment dwellers to 000 Buck for the rural crowd I'd just make sure it's something comercial.