A billet lower might have additional features like an adjusting screw to control the fit of the lower to the upper or an integrated trigger guard. The first might have some small impact on overall accuracy, the second is mostly cosmetic. Barrels are what make rifles accurate. A Crappy upper or lower might make a good barrel detectably worse, but the best billet set in the world will not make a bad barrel shoot.
I’ve never seen it, but I have heard tales of lowers that had the holes for the trigger pins drilled incorrectly or out of alignment. That would screw up trigger geometry and potentially contribute to poor accuracy. Aside from that, it’s hard to see how one lower could be significantly better than another.
