I wouldn’t remove the peep. It acts as your rear sight for sight alignment. I’d just work on making sure it’s lined up when you draw back or possibly use a larger size peep if you are using a smaller size.

Isaiah 6:8