A Libertarian Argument Against Opening Borders

The right of one person necessarily entails the obligation of another person or persons. If you have a right to life, I have the obligation not to kill you; if you have a right of free speech, I have the obligation not to stop you from speaking. The first half would be pointless without the second. If any given person has the right to enter the United States, who is it that has the obligation? Every person in the United States? Or only the person whose property the immigrant wishes to inhabit? Or perhaps, no one person in the United States, only the U.S. government, even if no individual in the U. S. wants him.