Unless mom otter thought her kids were in trouble I think the dog would be fine, unless the dog tried to attack the otter. In a fight between a dog and otter in the water I don't think the dog would have a chance, I have trapped and skinned quite a few otter, you will have a hard time finding an animal with a higher muscle density than the otter.

Some states used to allow you to shoot otters in the trap, this is a rare rule for animals that only allow harvest by trapping as it is hard to tell whether an animal shot in the head was actually trapped. I heard from a number of old trappers that the reason for that rule variance was that more trappers had been killed trying to dispatch otters caught in a leghold without a gun than any other animal. Skin a few otters and then tell me you don't think that is possible. Maybe no one has been killed by otters but a few looks at the teeth and muscles of an otter made me give them a lot of respect.

Maybe fact, maybe fiction, I don't know,
wade