It is going to take a lot more than mineral spirits, sadly... What happens is the solvents thin it out and you just keep chasing it. It will cause fisheyes and on a rubbed down stock the areas affected by the silicone may pull right off.
The best I have found commercially that is still not so nasty that it takes the finish off immediately is Formby's Furniture Restorer. It is designed to soften the finish and allow it to be moved around... Don't let it go that far!
After a couple good solid wipe-downs with mineral spirits and allowing it to dry after each rub the wood down with a rag just dampened with the Formby's.
It is just a mix of a lot of different solvents and mineral spirits is represented heavily... but it has worked well before.
Generally you do not need more oil, and a good coat of quality wax may be good enough for you. Keep quality wax on it and it will probably be just fine.
I like Bri-Wax real well, but Johnson's is acceptable, though soft.
