I don't make bottle caps but I use GIMP for my photo editing and I am certain you could make bottle caps there as well. There is a little bit of a learning curve to it but once you've played around with it for awhile you should be able to get the hang of it. GIMP is a free software found at http://www.gimp.org/
I know several of the other members use this program as well.
The easiest way to remove a background is to not have a background. A quick and dirty way of creating a white background for your photos would be to take a white piece of poster board/foam board/etc, set your item on top and take a photo from above. (I'm sure there are photographers out there cringing at that.)
There are a lot of free tutorials/how-to articles online about creating a lightbox. I personally made mine from white foam board, a piece of poster board, a couple of sheer curtains, and clamp lights with day light bulbs.