● When there is no entry in the ARP cache for the destination IP address of a packet, a broadcast is sent (ARP request, ARPOP_REQUEST: who has IP address x.y.z…). This is done by a method called arp_solicit().(net/ipv4/arp.c) – In IPv6, the parallel mechanism is called ND (Neighbor discovery) and is implemented as part of ICMPv6. – A multicast is sent in IPv6 (and not a broadcast).

● If there is no answer in time to this arp request, then we will end up with sending back an ICMP error (Destination Host Unreachable).

● This is done by arp_error_report() , which indirectly calls ipv4_link_failure() ; see net/ipv4/route.c.

● You can see the contents of the arp table by running: “cat /proc/net/arp” or by running the “arp” from a command line.

● In case it is for us, (RTN_LOCAL) we send and ARP reply. – arp_send(ARPOP_REPLY,ETH_P_ARP,sip,dev,tip,sha ,dev> dev_addr,sha); – We also update our arp table with the sender entry (ip/mac).

● Special case: ARP proxy server.

● In case we receive an ARP reply – (ARPOP_REPLY) –

We perform a lookup in the arp table. (by calling neigh_lookup()) – If we find an entry, we update the arp table by neigh_update().
● If there is no entry and there is NO support for unsolicited ARP we don't create an entry in the arp table. – Support for unsolicited ARP by setting /proc/sys/net/ipv4/conf/all/arp_accept to 1. – The corresponding macro is: IPV4_DEVCONF_ALL(ARP_ACCEPT)) – In older kernels, support for unsolicited ARP was done by: – CONFIG_IP_ACCEPT_UNSOLICITED_ARP Neighboring Subsystem – lookup
● Lookup in the neighboring subsystem is done via: neigh_lookup() parameters: – neigh_table (arp_tbl) – pkey (ip address, the primary_key of neighbour struct) – dev (net_device) – There are 2 wrappers: – neigh_lookup()

● just one more parameter: creat (a flag: to create a neighbor by neigh_create() or not))

● and neigh_lookup_errno()
Neighboring Subsystem – static entries
● Adding a static entry is done by:
arp -s ipAddress MacAddress
● Alternatively, this can be done by:
ip neigh add ipAddress dev eth0 lladdr MacAddress nud permanent
● The state (nud_state) of this entry will be NUD_PERMANENT
* ip neigh show will show it as PERMANENT.
● Why do we need PERMANENT entries ?
arp_bind_neighbour() method
● Suppose we are sending a packet to a host for the first time.
● a dst_entry is added to the routing cache by rt_intern_hash().
● We should know the L2 address of that host. – so rt_intern_hash() calls arp_bind_neighbour().
● only for RTN_UNICAST (not for multicast/broadcast). – arp_bind_neighbour(): net/ipv4/arp.c – dst→ neighbour=NULL, so it callsneigh_lookup_errno(). – There is no such entry in the arp table. – So we will create a neighbour with neigh_create() and add it to the arp table.

● neigh_create() creates a neighbour with NUD_NONE state

– setting nud_state to NUD_NONE is done in neigh_alloc()

The IFF_NOARP flag

● Disabling and enabling arp

● ifconfig eth1 -arp

– You will see the NOARP flag now in ifconfig a

● ifconfig eth1 arp (to enable arp of the device)

● In fact, this sets the IFF_NOARP flag of net_device.

● There are cases where the interface by default is with the IFF_NOARP flag (for example, ppp interface, see ppp_setup()(drivers/net/ppp_generic.c)

Changing IP address

● Suppose we try to set eth1 to an IP address of a different machine on the LAN:

● First, we will set an ip for eth1 in (in Fedora Core 8,for example)

● /etc/sysconfig/networkscripts/ifcfg-eth1

● … IPADDR=192.168.0.122 …

and than run:

●ifup eth1

● we will get:

Error, some other host already uses address 192.168.0.122.

● But:

● ifconfig eth0 192.168.0.122

● works ok !

● Why is it so ?

Duplicate Address Detection (DAD)

● Duplicate Address Detection mode (DAD)

● arping I eth0 D 192.168.0.10

– sends a broadcast packet whose source address is 0.0.0.0.

0.0.0.0 is not a valid IP address (for example, you cannot set an ip address to 0.0.0.0 with ifconfig)

● Garbage Collection – neigh_periodic_timer() – neigh_timer_handler() – neigh_periodic_timer() removes entires which are in NUD_FAILED state. This is done by setting dead to 1, and calling neigh_release(). The refcnt must be 1 to ensure no one else uses this neighbour. Also expired entries are removed.

● An example for setting LVS/DR on TCP port 80 with three real servers:

● ipvsadm C // clear the LVS table

● ipvsadm A t DirectorIPAddress:80

● ipvsadm -a t DirectorIPAddress:80 r RealServer1 g

● ipvsadm -a t DirectorIPAddress:80 r RealServer2 g

● ipvsadm -a t DirectorIPAddress:80 r RealServer3 g

● This example deals with tcp connections (for udp connection we should use u instead of t in the last 3 lines).

LVS example:

● ipvsadm -Ln // list the LVS table

● /proc/sys/net/ipv4/ip_forward should be set to 1

● In this example, packets sent to VIP will be sent to the load balancer; it will delegate them to the real server according to its scheduler. The dest MAC address in L2 header will be the MAC address of the real server to which the packet will be sent. The dest IP header will be VIP.

● This is done with NF_IP_LOCAL_IN.

ARPD – arp user space daemon

● ARPD is a user space daemon; it can be used if we want to remove some work from the kernel.

● The user space daemon is part of iproute2 (/misc/arpd.c)

● ARPD has support for negative entries and for dead hosts.

– The kernel arp code does NOT support these type of entries!

● The kernel by default is not compiled with ARPD support; we should set CONFIG_ARPD for using it:

● Networking Support→ Networking Options→ IP: ARP daemon support.

● see: /usr/share/doc/iproute2.6.22/arpd.ps (Alexey Kuznetsov).

● We should also set app_probes to a value greater than 0 by setting – /proc/sys/net/ipv4/neigh/eth0/app_solicit – This can be done also by the a (active_probes) parameter. – The value of this parameter tells how many ARP requests to send before that neighbour is considered dead.

● The k parameter tells the kernel not to send ARP broadcast; in such case, the arpd daemon is not only listening to ARP requests, but also send ARP broadcasts.

● Activation:

● arpd a 1 k eth0 &

● On some distros, you will get the error db_open: No such file or directory unless you simply run mkdir /var/lib/arpd/ before (for the arpd.db file).

● Pay attention: you can start arpd daemon when there is no support in the kernel(CONFIG_ARPD is not set).

● In this case you, arp packets are still caught by arpd daemon get_arp_pkt()

(misc/arpd.c)

● But you don't get messages from the kernel.

● get_arp_pkt() is not called.(misc/arpd.c)

● Tip: to check if CONFIG_ARPD is set, simply see if there are any results from

– cat /proc/kallsyms | grep neigh_app

Mac addresses

● MAC address (Media Access Control)

● According to specs, MAC address should be unique.

● The 3 first bytes specify a hw manufacturer of the card.

● Allocated by IANA.

There are exceptions to this rule.

– Ethernet HWaddr 00:16:3E:3F:6E:5D

ARPwatch (detect ARP cache poisoning)

● Changing MAC address can be as a result of some security attack (ARP cache poisoning).

● Arpwatch can help detect such an attack.

● Activation: arpwatch d i eth0 (output to stderr)

● Arpwatch keeps a table of ip/mac addresses and senses when there is a change.

● d is for redirecting the log to stderr (no syslog, no mail).

● In case someone changed MAC address on the same network, you will get a message like this: ARPwatch Example

● A neighbour in INCOMPLETE state does not have MAC address set yet (ha member of neighbour)

● So when neigh_resolve_output() is called, the neighbour state is changed to INCOMPLETE.

● When neigh_connected_output() is called, the MAC address of the neighbour is known; so we end up with calling dev_queue_xmit(), which calls the ndo_start_xmit() callback method of the NIC device driver.