First of all, a Colt 6720 is not a "cheap POS" rifle. They are the gold standard AR-15's. Second, you are talking about handloads, not standard, good quality factory ammo. You also didn't say if you used a scope or iron sights on your Colt.
Anybody with much time at a loading bench knows that handloads can make a big difference. E
6720 is a utility rifle with a chrome lined barrel. Made for work not accuracy.