This argument is related to the historical notion of secession and it comes in the form of a quote from Chapter 13 of Man, Economy, and State by Rothbard:
It is all the more curious, incidentally, that while laissez-faireists should by the logic of their position, be ardent believers in a single, unified world government, so that no one will live in a state of “anarchy” in relation to anyone else, they almost never are. And once one concedes that a single world government is not necessary, then where does one logically stop at the permissibility of separate states? If Canada and the United States can be separate nations without being denounced as being in a state of impermissible “anarchy,” why may not the South secede from the United States? New York State from the Union? New York City from the state? Why may not Manhattan secede? Each neighborhood? Each block? Each house? Each person? But, of course, if each person may secede from government, we have virtually arrived at the purely free society, where defense is supplied along with all other services by the free market and where the invasive State has ceased to exist.