Thanks for all the advice.
I am a log builder I have thousands of dollars of tools. I have 2 of the 7 inch disc grinders, a DeWalt and a Makita. I have a real nice Makita belt sander. All kinds of expensive drills etc etc.
But I don't have a dremel tool. Never used one.



[Linked Image from i.imgur.com]
I went down to Lowes and bought this dremel grinding stone for $3.50. It fit right in to my DeWalt cordless drill.



[Linked Image from i.imgur.com]
This thing worked great! I knocked out that little rim in about 7 minutes.

Thanks for all the advice. I did buy some 80 grit and 120 grit sandpaper and was going to wrap it around the ramrod to my Kentucky rifle, and duct tape it on. But it didn't come to that the home-made dremel tool did the job. Easy.