" I pretty calmly signaled him to come back and chat..."

That was your mistake. You need to ignore scheisskopfs like this guy.
You gonna wind up on trial for second degree homicide, because you reacted when some jerk flipped you a bird?

No, my friend, just ignore the buffoon, and stay away from trouble, if you can.