It's kind of hard to do something different after the fact.
The painter works for you...you are the employer. If you want him to stop work, you tell him to stop. If an employee doesn't follow orders, it's your responsibility to take control.
The normal course of business is to withhold payment until the job is completed to your satisfaction. Your final payment then represents your final approval and acceptance of the results of the project. The end result is mostly on you.
But regardless of how you got here, I'd recommend to have it repainted the way you want it and do it right away before the new paint weathers. It seems unreasonable at this point to expect the contractor to redo the work for free after you signed off on it and paid in full unless he just feels like doing so as a customer satisfaction policy.
The siding is not a "bad" color, but everyone sees color differently. I may not like the siding color either, and whoever mixed the paint may not have been awake that day, so I might suggest applying an accent color to the siding and the fascia. It shouldn't cost much to repaint the siding and trim.
If a painting project is done right, 90% of the work is in preparation of the surface and only the last 10% is actually applying the paint. It takes me 2 weeks to prep a house and only a day to spray and brush.
I'd recommend to tell your contractor your issues with the finished project and see if he offers you some concessions. It should only take a day's labor to repaint the siding and the trim plus the cost of the paint.
Considering that this issue is mostly your responsibility, if he's willing to split that with you he still might be a decent fellow.
