I've shot perhaps a dozen deer total with handguns and been able to examine the results of handgun shots on about that many more.
My observations lead me to this:
1) For cartridges under .40 caliber, I want a jacketed bullet.
2) At .40 caliber, I think the potential is there for cast bullets, but I have not seen .40 cal cast bullet with a wide enough meplat yet, so I'll stick to jacketed bullets for now.
3) From .41 to .45, I like cast bullets better IF they are of a wide nose or fairly blunt SWC design. Some of the SWCs with a smaller, pointier nose do not work real well. A blunt SWC or one of the LBT-type designs kills just as fast as a jacketed bullet but without so much meat loss.
I haven't tried one of the above .45 cal cartridges yet.
Tom