Your using the best equipment offered in your dies. Your brass will tell you when it needs to be annealed. You'll notice this by the amount of springback when resizing. If you are finding it difficult to resize, it's probably time to anneal. I usually anneal my cases every 3-5 firings. At the end of the day, annealing addresses two issues 1) prevents premature neck splits from work-hardening and 2) creates consistent neck tension through the life of your brass. In annealing the last 12 years, I can say that annealing has maintained consistent accuracy throughout the life of your brass. If not annealing, you find that the first 1-3 firings are really consistent. After that, your groups will start to open up.
If you want to try annealing on the cheap, try something like this and test for yourself:
https://www.midwayusa.com/product/360902/hornady-annealing-system
