It was actually debated quite a bit when the Constitution replaced the Articles of Confederation.
https://en.wikipedia.org/wiki/Secession_in_the_United_StatesThe federalists acknowledged that national sovereignty would be transferred by the new Constitution to the whole of the American people—indeed, regard the expression, "We the people ...". They argued, however, that Henry exaggerated the extent to which a consolidated government was being created and that the states would serve a vital role within the new republic even though their national sovereignty was ending. Tellingly, on the matter of whether states retained a right to unilaterally secede from the United States, the federalists made it clear that no such right would exist under the Constitution.
Thus Madison affirms an extraconstitutional right to revolt against conditions of "intolerable oppression"; but if the case cannot be made (that such conditions exist), then he rejects secession—as a violation of the Constitution.
