I had the exact same experience with a Remington VSSF II .204 Ruger a few years ago. I was so disgusted with Remington that I sold the gun for parts. I got the same story as yours from their customer service folks. The Remington certified gunsmiths in Des Moines shot the gun, confirmed it shot as I said, then proceeded to tell me it was "good enough" and that I should learn to shoot better. I was positively livid.
The best mine would shoot at 100 yards was 2" including factory and handloads. It shot 39 and 40 grainers sideways through the target. The prettiest key-holes you can envision.
Sadly I sold an original VSSF in .223 that shot sub 1/2" handloads all day long years earlier.
After parting out the Remington I bought a Cooper. Funny they don't tell you they shoot that amazing little target at 33 yards in a tunnel with a 36X scope. At least that's what they told me when I inquired and asked what the powder charge was on their test load. It did shoot a couple groups right at 1/2" but 5/8" was the norm. It got sold.
I now have a custom Kreiger-Stiller-McMillan-Jewell-Nightforce contraption. I went 1/10 on the twist and used a match (short leade) reamer. It drills one hole 5 shot groups with boring regularity. It's more fun when it's hard to miss :-).