I don't know about a rule of thumb, but many if not most states have minimum requirements for firearms used to hunt big game, and those requirements are based on the kinetic energy of the bullet fired.

I would gladly use a .25-06 rifle to hunt deer, but would consider it a poor choice for a self defense weapon, so what does this prove?