AirPlay, VLANs, and an Open Source Solution

As I’ve written about in the past (here), Apple’s AirPlay technology relies on Bonjour which is Apple’s implementation of “zero config” networking. One of the things that Bonjour enables is the automatic discovery of services on the network. For example, an Apple TV might advertise itself as being able to receive AirPlay streams. An iPad that is looking for AirPlay receivers would use Bonjour to discover the Apple TV and present it to the user as an AirPlay destination. Both the Apple TV and iPad do all this without any user intervention or configuration (hence the “zero config” part).

That’s fine and dandy but what my earlier article focused on was how Bonjour broke down in a network where what I’ll call the “server” and the “client” are not in the same Layer 2 domain/VLAN. This is because the service discovery aspect of Bonjour relies on link-local scope multicast. These packets will not cross Layer 3 boundaries in the network.

Bonjour packets will not pass a Layer 3 boundary

What’s needed to make Bonjour work across subnets is a proxy that can take the service announcements on one subnet and announce them on the other(s) (and vice-versa). What’s perfect about this is that service discovery works just like DNS. In fact, it is DNS: multicast DNS (mDNS). The DNS system provides a lookup service that sits out of band of the actual traffic flow. mDNS is exactly the same. The mDNS proxy will need a leg into each subnet where AirPlay clients and servers live, but it does not have to relay traffic between the subnets. It’s merely the lookup mechanism. The reason I’m stressing this concept is because it means that the mDNS proxy can be deployed in the network without changing the network architecture. It also does nothing to change security zoning, doesn’t affect the resiliency of the network and doesn’t create a bottleneck for network traffic. It’s a very simple way to address the need (demand?) for AirPlay in a business network. Eventually all the Wi-Fi vendors will have this kind of thing built into their products but for now, this gives the IT Network team the ability to say “yes, our network can support AirPlay”.

The software I’ve used as an mDNS proxy is called Avahi. It appears to be the defacto open source software for service discovery and mDNS services. Yeah that’s right, you can use it to advertise the services that are running on the local box (SSH, HTTP, etc). But what’s really interesting is the setting that enables the “reflector” functionality.

[reflector]
enable-reflector=yes

This little knob in avahi-daemon.conf turns Avahi into an mDNS proxy where it will pick up service advertisements received on one interface and send them out on any other interface where a discovery query is received. The list of participating interfaces can be restricted using the allow-interfaces knob.

[server]
allow-interfaces=vlan10,vlan20

Bonjour packet flow with Avahi

As the diagram illustrates, this is the packet flow now between the iPad and the Apple TV:

Apple TV advertises itself as an AirPlay receiver using Bonjour’s Service Discovery mechanism

Since the Avahi server is connected to the same VLAN as the Apple TV, it hears the advertisement and stores the details in its database

The iPad uses Service Discovery to try and locate an AirPlay receiver on the network

Since the Avahi server is also connected to the same VLAN as the iPad, it hears the discovery and responds with the information for the Apple TV

Within the response from Avahi, the iPad will learn the IP address and port that it should connect to the Apple TV on. The iPad will initiate a direct connection to the Apple TV

Avahi is a really lightweight piece of software. It can run on a small UNIX machine — physical or virtual — and handle discovery messages on multiple VLANs. Avahi is present in all the major Linux distributions and in the OpenBSD and FreeBSD ports collections.

Go run it.

Disclaimer:
The opinions and information expressed in this blog article are my own and not necessarily those of Cisco Systems.

Thank you for this excellent solution. I implemented this at a High School segmented into 12 wired VLANs and 4 wireless. Since I am using Microsoft Hyper-V to run Avahi, I had to create two Avahi servers to connect all of the VLANs (Hyper-V) has a limitation of 12 virtual ethernet interfaces per VM). Now everything works great, but I fear that the two servers are echoing each other’s broadcasts, since they share the wireless VLANs. What can be done to remedy this?

Hey Mike, thanks for the feedback. Have you considered giving your VM a single vNIC (sorry, I don’t know the HyperV term for the guest NIC) and tagging your VLANs all the way to the VM? You would end up with a VM-on-a-stick instead of a multi-legged VM.

Yes, a single NIC would work as long as you’re able to do VLAN tagging (802.1Q) on the NIC of your Avahi box. You’d create one VLAN interface for each of the VLANs you want to exchange Bonjour messages between. The configuration for this is going to be up to whatever OS you’re running; they’re all different.

Hi Joel, this is an awesome post! I tried using the idea and apply it to VPN. I have a device with two NICs and also acts as a router gateway and VPN server.

I have avahi running properly on the device with reflecting turned on. I also see in the logs that avahi successfully registers eth0 (internet), eth1(lan) and ppp0 (my machine VPNed in) for mDNS pooling.

But still no dice! iOS device that is in eth1’s network cannot see the AirServer I have running on my laptop that is VPNed in through ppp0.

Just as extra info:
my laptop can ping addresses in eth1.
addresses in eth1 have internet traffic routed properly through eth0.
VPN technology is PPTP.
System logs says that avahi says: “avahi-daemon[1540]: Invalid query packet.” Is that the query from the iOS device asking for an AirPlay server?

In any case, I’m at the end of my wits now and was hoping maybe you could give a tip or two to get through this!

What does the output of “avahi-browse -at” look like? (I think that’s the right command. Maybe avahi-browse -ac?)

My first thought is that multicast is not passing on your PPTP link and Avahi has nothing in its cache to reflect on the eth1 LAN interface. If memory serves, I too get that invalid query packet message from time to time. I chalked that up to Avahi falling behind a little bit with respect to the current versions of the AirPlay protocol, but I never chased it down or dug into it.

I think your hunch is valid, before and after I connected via PPTP VPN the avahi-browse -ac and -at both showed only devices from the eth1 interface, no devices from ppp0.

This is despite the fact that after connecting via PPTP VPN the system logs shows this:

Joining mDNS multicast group on interface ppp0.IPv4 with address 192.168.1.37.
daemon.info avahi-daemon[2007]: New relevant interface ppp0.IPv4 for mDNS.
daemon.info avahi-daemon[2007]: Registering new address record for 192.168.1.37 on ppp0.IPv4.

To test your hunch, I will VPN my iPhone which for sure broadcasts AirPlay multicast (as opposed to AirServer which is a 3rd party app on my macbook pro) and then re-examine the avahi-browse -ac, -at outputs… OK just did the test and same results, outputs only show eth1 devices.

Not sure if relevant but when connecting through PPTP VPN the IP address that the devices is configured to give to the connecting client (on pppX) is from the same subnet that eth1 has (192.168.1.X)

Yes that does seem to be the case. Kind of discouraging since the config file has an option for allow-point-to-point and I set it to ‘yes’ which resulted in it adding and registering the ppp0 interface.

No idea why it would not go just one more step and actually hear the PPP broadcast. Maybe its a setting on my laptop I need to look at then, if it is choosing not to broadcast on a ppp interface

Quick question: Would this work for 3 VLANs? For example if I have VLANs 1,2,3. Traffic is enabled between VLANs 1 and 2, and 1 and 3 (2 & 3 can’t communicate). Can I put the RPi on all three VLANs and airplay from devices on VLAN 2 or 3 to a receiver on VLAN 1? Thanks!

I’m not sure what you mean by RPi so I hope I’m not confusing things, but yes, you can run Avahi on all three VLANs and services will be reflected from each to the other two.

I’m not sure if it’s possible to tell Avahi not to reflect between VLANs 2 and 3. By default, if you have a player in VLAN 2 and 3 and you also had a receiver in say VLAN 3, that receiver would see the player in VLAN 2 as well as the player in VLAN 3. That might be confusing to users if you don’t allow traffic from 3->2.

I think this would be possible with 2 Avahi proxies – one between VLAN 1 & 2, and one between VLAN 2 & 3.

We’re planning on trying something similar for security. Basically, we want to set up an “Airplay DMZ” that will allow people from the “Office Network” and people from the “Wireless Guest Network” to both see and Airplay to the same set of AppleTVs. In our case, the “AirPlay DMZ” would be VLAN 2, with VLAN 1 and VLAN 3 not being able to see or reach resources in each other.

Wouldn’t the two Avahi boxes connected to VLAN 2 end up reflecting services from their non-DMZ VLANs to each other?

I’m struggling to remember all the details but if you’re a Cisco shop, IOS and the WLC both have features in them for selectively allowing Bonjour between VLANs. It may have enough features to allow you to do what you want natively in the network. You probably want to look at fairly recent code.

I’ve gotten Avahi working to enable AirPlay and Mac OS X printer sharing among 3 VLANs. It works fine except for one extremely annoying problem: intermittently, Apple devices will decide that there’s another machine on the network with the same name, so they change their name by adding a number at the end: “mymac” becomes “mymac (2),” or “Fred’s Apple TV” becomes “Fred’s Apple TV (2)”. Even the name of a shared printer on a Mac OS X Server machine may change, from “hp printer @ macserver” to “hp printer @ macserver (2)”; if you’ve added a printer on a Mac with the original name, then if the printer’s Bonjour name changes, you can no longer print to it. Any idea why this might be happening and what to do to stop it?

This happens to me too and I’m not sure what causes it. My assumption is that it’s Avahi doing it because it believes there’s two devices on the network with the same name so it adds a number the the newest one. In my network, this happens a lot with a particular Apple TV that gets powered on and off a lot. I wonder if restarting Avahi or even just clearing its cache would resolve this?

It would be a cosmetic issue, except that it renders OS X shared printers unusable. I’ve assumed it was the devices changing their own Bonjour names, since just stopping Avahi doesn’t resolve it. My Apple TVs and Mac laptops that have added numbers to their names have held onto the new names, and I’ve had to stop Avahi and then restart print sharing in OS X Server to get a shared printer to revert to its original name. I’ve seen a few forum posts saying it’s a bug in Avahi, and nothing’s been done to Avahi since early 2012… I’m getting desperate for some kind of solution that actually works. I tried setting up mdns on an AD DNS server, but it’s a pain to set up, and once I started publishing services, OS X clients seemed to go blind to everything except what’s advertised on the DNS server.

Well…maybe I’m jumping the gun, but it seems to be behaving now, after setting cache-entries-max=0 in /etc/avahi/avahi-daemon.conf. That kind of makes sense, since we don’t want it to cache anything, just reflect. I was able to trigger the problem by printing to a shared printer from a Mac with 2 network interfaces connected to 2 Avahi-enabled VLANs at the same time. Now with cache-entries-max=0 in place, I can no longer get the problem to happen. We’ll see if that did it. Thanks for the cache hint!!!

Hmm, I didn’t notice that, but my only objective is connecting iPads, MacBook Pros, Apple TVs, and Mac-shared printers across subnets. With caching turned off, Avahi seems to be working perfectly for that now, and no longer doing any of the destructive things it was doing before.

Thanks a lot for your article! I was trying to get AirPlay working between VLANs today, but I didn’t succeed. I’m sure I just forgot something… Here’s my setup:

Ubuntu 14.04 VM on ESXi host, 2 virtual NICs (one on std. VLAN 1 [192.168.111.0/24], the other one on VLAN 20 [10.20.0.0/24]). Avahi and Avahi-Utils installed, reflecting is on and allowed-interfaces is set to eth0 and eth1. The VM is reachable from within both subnets and gets its addresses via dhcp from our watchguard router.

We have an Apple TV on VLAN 20, and so far only clients in the same VLAN can successfully connect to it. When in VLAN 1, clients can discover and “see” the Apple TV on VLAN 20, thanks to the VM, but it errors out when they try to connect to it.

Any clues? Do I need to set up routing for the actual traffic between the subnets?

Is the Watchguard the default gateway for hosts in vlans 1 and 20? If so, I suspect that the firewall policy is not allowing the Airplay traffic to cross between the VLANs.

It can be a little confusing, but the way this works is the Avahi box reflects the Airplay announcements between the VLANs (thereby bypassing the firewall). Then when an iDevice tries to connect to an Airplay source, that traffic is actually sent towards the Watchguard (and not through the Ubuntu host). Because of that, the Watchguard will need to be configured to allow Airplay between the VLANs.

If you are using WLC version 7.4 or later you do not need a external server like Avahi. The WLC can act as Bonjour gateway and provide the services to devices in different subnets using the mDNS snooping feature.

just been reading your article,
So just to make sure i have this down – i have a
juniper Router with 4 subnets each on its own interface port.
10.0.1.x/10.0.2.x/10.03.x/10.04.x each has its own switch with say 5 Macs on them each.
Right now when i browse network on any of these subnets i can only see the other 4 machines on the network.
My firewall allows full sharing between all the subnets so i goto connect to and type in afp or nfs and the address I can share with any other computer.

In order for me to share bonjour/Mdns information on a Mac in each subnet, I could install Avahi on say a server with Ubuntu, I would need to have configured 4 separate network connections going to each subnet switch.
Then i would enable the Reflector as your command showed, and technically now, any mac on any of those four subnets should be able to see all macs on the network via the network tab? As well as discover printers on any of the subnets?

1. Your Ubuntu box doesn’t necessarily need 4 physical connections; it could be 4 virtual NICs in the case of a VM or, it could be 1 physical NIC port where you just trunk each of the networks into the box.
2. As the article says, the traffic from one Mac to another will still pass through your router. The Ubuntu box will just pass the mDNS messages between the networks; it does not act as the gateway between the networks.

Have a minix (with airpin services) box in ore core VLAN and uses that
connect to a guest-wifi vlan should be able to connect or mirror their laptop with the beamer where the andriod minix with airpin service.

Have installed avahi on ore pfsense box so multicast traffic could proxy through VLAN`s.

When i do a *tcpdump -i em1_vlan31 -s0 -vvv net 224.0.0/4 I`m seeing
multicast traffic on the guest-wifi vlan. * i think.

I see your tcpdump output and looks like the mDNS packets are being received by the pfSense box.

Have you configured firewall rules on the pfSense box to allow multicast traffic in on your guest-wifi and core interfaces? If the firewall is blocking multicast traffic, then avahi won’t see those packets and will have nothing to reflect.

– Your wireshark output is showing TCP traffic. I’m not sure how this relates to mDNS or how it’s relevant.
– The multicast packet in your wireshark is for group 239.255.255.250. That’s not the group that mDNS uses. Avahi is not going to do anything with that group.
– You mentioned Android Minix. I don’t know what that is, but are absolutely certain it’s using AirPlay? The open source AirPlay libraries that I’ve seen do not work with current versions of iOS. Could that be part of the problem here?

Great blog post and amazing how you are keeping up with all the comments. Thank you! I wonder if you can help me:

I have some Raspberry Pis on a corporate network running the avahi-daemon.
Ethernet address range is 10.216.20.xx
WLAN address range is 10.216.30.xx
Can I use avahi to bridge those and publish services across the two networks?
When connected to the WLAN with my laptop I can ping the IP addresses of the raspberries on ethernet but I can’t ping them via hostname.local. https://www.evernote.com/l/AAV7Z7cr5W9C-5Ok1L59_O4SvspCePG5JhM

I have one raspberry (old-raspberry) connected to the WLAN with
[reflector]
enable-reflector=yes
configured. Strangely I can ping that raspberry’s IP address from my laptop but I can’t ping it’s hostname.local either despite being on the same network.
Any tips on what else I should try?

I don’t know if I would trust ping for testing that mDNS is working. It assumes that mDNS is being used to resolve hostnames to an IP address. Do you know if your OS is actually using mDNS in this way?

I suggest you connect an iDevice to the WLAN and see if you can see any AirPlay receivers. I’ve also used the HE.net Network Tools app (free in the App Store) which has a built-in Bonjour Browser. This will let you see anything and everything that’s announcing itself via Bonjour, including the devices on your LAN if Avahi is doing its thing correctly.

Hi, your explanation is good. Do you have any idea/ resources on how a sender communicates with receiver in a detailed manner. I really need a software architecture of Airplay in a diagram form but I haven’t found one. Thanks in advance

Hi Ror, thanks for the feedback. As far as I know, Airplay is proprietary and there’s no official architecture document that describes it in detail. There are open source libraries though that have implemented Airplay. A quick Google shows https://github.com/jamesdlow/open-airplay as an example. Perhaps they have some data about how the protocol works. Worst case, you can read the code.