I take a small carry-on back and forth to work also. When traveling on vacation, I do the same....and buy clothes when I get where I'm going, then pitch 'em in the trash on the way home.
Was surprised that last time I flew south from Anch, I had 200 rounds of brass and 200 TSX's in my carry on....they ran it through the Xray twice, but never said a word, and didn't take any of it away......
