Ranger,

Don't flux frequently. Just flux once the alloy is molten. Fluxing frequently while casting causes weight variances.
I only questioned the frequency of your fluxing, since you are having weight fluctuations.

How are you sizing your bullets? What diameter die? Are your seating dies deforming your bullets as you're loading them?

A cast bullet (especially a gas check design), sized base first with the gas check swaged on first as it enters the sizer die, will make unconcentric bullets even with an aligned fitted nose punch.
A good way to detect this misalignment is seeing whether the driving bands are showing thicker on one side than the opposing side. Using a larger die and sizing nose first and mildly swaging
the gas check last provides more labor and expense, but also provides the consistent accuracy you seek. Lubing/final base first sizing can take place after the check has been seated using the nose first process. Use a steel punch for nose first sizing;it will ruin your nose fitted aluminum one.
Even most cheap, mass produced cast bullets are usually nose first sized on Star equipment.
An example for a .458 throat uses a .460-.461 die, sizing nose first until the check is mildly crimped on. It might be loose, but that won't hurt a bit. If you get any galling or shaving, lubing a bit with case lube like Imperial wax works well. Lubing and final sizing is done with a .458 die with an aligned fitted nose punch.

I use a Star sizer, so all sizing and lubing is done nose first.

Said another way, lopsided bullets are never accurate and alloy, diameter, weight and hardness won't fix the issue. The bullets MUST be concentric.

I thought you were powder coating, so I suggest NOT sizing and NOT using a gas check for now (I also thought you were using checks, so correct me if I'm wrong!). Seat as far out as will chamber with fouling, snug. Apply a mild roll crimp if not in the crimp groove. We don't want bullets setting back at all. If you can still crimp in the groove, swedge the crimp snug to the bottom.

The dimensions of your bullets, before and after loading and how it fits in your rifle's throat, will matter a lot more than the weight of the projectile and even the hardness of the alloy, within reason.