I wouldn’t remove the peep. It acts as your rear sight for sight alignment. I’d just work on making sure it’s lined up when you draw back or possibly use a larger size peep if you are using a smaller size.
+1 For accurate aiming and consistency a peep is critical. You can go with a very large peep and circular pin housing and be very accurate as long as you keep things centered. Good strings work wonders in keeping a peep sight aligned at full draw.
One other note is to have your peep properly located. You should be able to draw and anchor with your eyes closed - when opened you should be looking through the center of the peep. If not move your peep.
