Were our nation governed by non-criminals, this would never become a relevant question, because the instant the authorities realized they were dealing with an illegal alien, he'd be in custody till they could return him to his nation of origin, and the carrying of firearms while in custody has always been illegal in the US.