It really depends on the construction of the wall. Tree roots will always seek out water and will most likely move in that direction. If they hit something solid they will reroute themselves, but over time that root will gain girth and most likely push your wall. I plant alot of birches, but I usually try to keep them away from walls. That should go for ANY tree that eclipses 20'

I love to do mass plantings of Liriope or daylillies w/ Birches. I also like Blue rugs around the tree to add some evergreen presence. Other shrubs I will use are Goldthreads, Inkberry Hollies, or Microbiota. Don't forget some nice boulders to go with that Birch.

If you really want trees that close to the wall can you change the wall material? Root issues would be minimal if you construct the wall with boulders. Otherwise, 4' is too close, especially if a birch has sufficient irrigation. A river birch can grow extremely fast but can also be short lived. It is also one of my favorite trees.