All the cards that are paid off cut them up and cancel the accounts. Keep one for emergencies with the highest credit limit or the one that's an Airline card you can use it if the purchase will literally save your life or a family members.
And one solution I've seen used was to take that card and freeze it in a block of ice. It's still useable but takes some planning
