And the bullet will maintain its alignment from the point of firing thanks to the spin - firing at any angle apart from straight up means
it will be travelling sideways downrange and be subject to tumbling as a result.
At no point in time does the bullet "travel sideways".
The spin keeps the bullet spinning on a fixed axis -
it could be going sideways down range, or if disturbed it will tumble out of control.
The spin lasts a long time - in most cases it's still spinning when it hits the deck. There's minimal resistance otherwise to slow the spin. There's video of 9mm bullets fired into ice and they keep spinning. Mythbusters did this too.
Repeating that won't make is so.
The projectile remains aligned point first with it's path of travel during it's entire arc.
They never go "sideways".
That is not necessarily the case. For example, a bullet which is overstabilised will tend to remain aligned with its angle of departure, rather than "nosing over" to follow the trajectory as we get out to very long range. There are also a number of forces at play tending to cause a bullet to yaw in flight, and not just at very long range, and bullets will tend to drift sideways as a result of wind and spin drift, among other things.
There's a rather good treatment of these questions a fellow named Ruprecht Nennstiel put together a while back :
http://www.nennstiel-ruprecht.de/bullfly/
