Look at it this way: An alternate universe is a physical location. Matter vibrates at different frequencies, quantum fluxes in the cellular RNA, or whatever the hell it is. Think of the DC multiverse, for example. Read "Crisis on Infinite Earths."

Except that doesn't make much sense. Here's what I said on the subject in a blog post:

[T]he idea [is] that if the multiverse is infinite, if there’s an infinite number of other universes alongside ours, then probability demands that some of them will be exact duplicates of our universe, just happening by random chance to have the exact same combination of particles and thus producing the same galaxies, stars, planets, species, inviduals, etc. — kinda like how the famous infinite number of monkeys banging on an infinite number of typewriters will inevitably produce all great literature by chance. Thus, so the claim went, there could be other universes out there that are essentially parallels to our own with duplicates of ourselves, except maybe for some minor variations.
...
But... I have my problems with the reasoning employed. For one thing, it’s purely an ad hoc assumption that the multiverse is infinite rather than finite. If it’s finite, then there’s no guarantee that there would be other universes that exactly duplicate ours. Certainly there could be ones with compatible physical laws, with their own stars and galaxies and planets and life forms, but odds are they’d be different planets, different species, different individuals. No duplicate Earth, no duplicate Lincoln or Kennedy or Jet Li.

And if the multiverse is infinite, then sure, you could argue that with an infinite number of tries, it’s inevitable that our universe would be exactly duplicated somewhere. But the flip side to that argument is that if there’s an infinite number of universes, then the odds that any given universe would duplicate ours would be n divided by infinity, or effectively zero. In practical terms, if we found a way to visit other universes via wormholes or something, then we could search for an infinite amount of time before finding one that had its own Earth and human race and history duplicating ours except for having more goatees or whatever. Thus, by any realistic standard, such duplicates would be effectively nonexistent.

And sure, you could fudge that for fiction, but here's what I said on why I don't find it useful or desirable to do so:

If these universes are just infinitely separated samples of an infinitely expanding metacosmos, then that doesn’t really unify them in any way, does it? They’re so far apart, so mutually unreachable, that the “connection” doesn’t really count as a connection at all. (After all, given the underlying physical premise, there’s no realistic chance of any kind of wormhole link or inter-universe crossover anyway.) It’s a trivial and useless result fictionally for the same reasons it is physically. And if they’re specks in an infinite sea of universes, it makes them all feel kind of irrelevant anyway. So why even bother? It’s simpler just to treat them as distinct fictional constructs and not bother trying to unify them. Besides, even if I know intellectually that the humanity and Earth and Milky Way of my fictional universes aren’t the same as my own, it’s more satisfying to pretend they are, to construct a satisfying illusion for the readers that they’re reading about an outgrowth of our own reality, than to pretend that they’re some totally separate duplicates in universes unreachably distant from ours. No point going out of my way to create a premise that alienates me and my audience from the universes they’re reading about.