You don't say what powder you're using, but with black, I don't expect to get more than a few shots off before having to wipe the barrel. I use Ballistol with BP and 777, either diluted or straight. I pick the nipple every time before capping. Consistent pressure on the bullet also matters, I think, but I don't bounce the ramrod like some, just push on it firmly.
Wiping the barrel out with dry patches, then popping a few caps should get rid of oil or moisture.
Find what works with your setup then stick with it.