And the bullet will maintain its alignment from the point of firing thanks to the spin - firing at any angle apart from straight up means
it will be travelling sideways downrange and be subject to tumbling as a result.
At no point in time does the bullet "travel sideways".
The spin keeps the bullet spinning on a fixed axis -
it could be going sideways down range, or if disturbed it will tumble out of control.
The spin lasts a long time - in most cases it's still spinning when it hits the deck. There's minimal resistance otherwise to slow the spin. There's video of 9mm bullets fired into ice and they keep spinning. Mythbusters did this too.