Should be noted that due to the pressure of the air, you're base won't become unbreathable instantly. Indeed, if you have enough atmosphere generation, you could maintain the leak. (Perhaps using it to trap alien fauna)

How does it react to things like caverns with a corner tile that is open environment?

As due to how the tiles are set up, open outer corners actually create open spaces between the walls and it would kind of suck to start up new games only to find your internal environment is already exposed because one of the default open spaces has a outside corner tile that is outside, or to be digging inside only to find what should have been a safe move for some odd reason ended up exposing your entire base.

Example:

That's the result of removing one wall piece from inside, which automatically removed the bridge between the two walls opening up the interior to the outside world.