With a brass punch it's like - tap tap tap ... doesn't move.
Tap tap tap ... moves too far.
With this method I can get it close. For fine adjustment I'm going to try the Williams sight pusher. I think that should work good.