I've killed a few deer with the 38wcf and it worked fine. With a good bullet I think the 40 would be fine if you choose your shots. I have always used a soft point on deer, but have loaded 180 grain hollow points and have some consern that they may be a little extreme for deer a rifle speeds.
Deer aren't bullet proof.