Wi-Fi Issues During RIPE 65

Band steering

The new wireless network SSID we used for the recent RIPE 65 meeting in Amsterdam was configured to use band steering . This is used to encourage users to prefer a 5GHz network, which has many more channels to play with. This seems to work fine with both Windows and Mac clients; unfortunately, wpa_supplicant (Linux & FreeBSD) does not support band steering. There is a Redhat bug report and a post discussing the issue on a Debian platform .

Band steering causes an access point to silently drop probe requests on the 2.4GHz network, if it knows a client is capable of 5GHz. If wpa_supplicant decides that a 2.4GHz network BSSID is the best thing to connect to, it will try to connect to it. When it times out, it retries 3 times, blacklists the Access Point (AP) and tries to connect to another one. It does not try the same BSSID over 5GHz, whereas the band steering setup assumes it will.

During the RIPE 65 meeting, there were 468 attendees and we had a peak of 500 wireless clients on the wireless network. 7% of these were Linux users who could potentially have had problems. In the end only 7 attendees reported problems with their wireless connectivity.

SSID without band steering

In an effort to solve these issues, we set up a second SSID named "ripe65 no bandsteering" (sorry for the whitespace). This was a copy of the RIPE meeting SSID (ripemtg) with band steering turned off. We did not announce the configuration of this new SSID, because the attendee assisting us in trouble shooting hit another problem: association was much quicker but handing clients over to different APs seemed to take a significant amount of time. When a client lost its connection to an AP, it first tried to connect to the lost AP three times before re-electing. The timeout between each attempt was significant. The total handover period took about 40-50 seconds.

We were unable to recreate the issue on our own hardware using live CDs of Ubuntu LTS 10.04.4 and 12.04.1. However, it would appear that the advice described in the Debian post would allow users to use band steering. Specifically using nl80211 with a recent version of wpa_suplicant and modifying the necessary dbus rule . This is not ideal as it means building external packages and updating standard configurations. So we still wanted to get the "ripe65 no bandsteering" SSID to work.

We have details of the hardware, kernel and driver used in the machine which had the issues described above. We will try and recreate this environment in the hope that we can recreate the issues observed, and have them fixed before the RIPE 66 meeting in May 2013.

Further investigation

While troubleshooting this issue, we also noticed other strange behaviour. One Linux machine sent a probe request for the  ripemtg SSID to a specific BSSID. However, it did not receive a response for that probe; at least it did not receive one visible to wpa_supplicant. When we looked at this in the AP management software, it seemed as though more than one response had been sent, but from different BSSIDs.

Unfortunately we did not have time to investigate this issue thoroughly during the meeting. We will attempt to recreate this issue and work with the vendor in an effort to understand if this is expected behaviour and whether it is standards-compliant. Once we have something more concrete, we will provide an update.

Other issues

As mentioned in the closing plenary, we bought a new hardware platform,  Aerohive . We also added 802.11n support to the wireless network. We are very happy with this choice. However, as with anything new, there is always a learning curve.

The new network has a very dense deployment of APs, with roughly 60% more APs deployed than at previous meetings. At the beginning of the week we configured all APs with both 2.4GHz and 5GHz networks. However due to the narrow channel distribution of the 2.4GHz network, we noticed signal interference, even with the radio power turned to its lowest setting. In order to rectify this, we disabled the 2.4GHz network on 50% of the APs. As we were using band steering, the 2.4GHz network was fairly underutilised so this did not cause any contention.

In previous meetings we created channel maps and set the channel of each AP manually to overcome interference. The Aerohive technology supports Dynamic Frequency Selection (DFS), allowing the APs to be automatically configured with a channel that has the least amount of interference. With 802.11n DFS also allows APs to utilise the Unlicensed National Information Infrastructure (U-NII) radio band. At some point during the meeting, DFS stopped working and we noticed a few areas where all the APs were on the same channel. This resulted in packet loss and latency.

We are unsure why DFS stopped working and will follow this up with the vendor. Our theory is that there may be problems with pushing configuration updates to a large number of clients. We resolved this issue by pushing the configuration out to the APs again, but in batches of 10.

Input needed

Unfortunately we were not aware of the issue with band steering until the middle of the RIPE meeting week. For future meetings, we intend to bring multiple laptops with different wireless chip-sets.  We will have these machines set up to dual boot the current versions of Windows and Linux (probably Ubuntu LTS). If desired, we can also set up FreeBSD.

We noticed that attendees seemed reluctant to report issues to the RIPE NCC operations team during the RIPE meeting. We would like to encourage you all to come and talk to us if you experience any problem. We run a busy network and try to implement state-of-the-art technology. At the same time, we need to ensure everything works on a wide range of devices and operating systems. Unfortunately we do not have the resources to test all of these combinations before the meeting starts. We rely on reports provided by our users so we can try to resolve any issues. We also recognise that we have experts in our audience and we would welcome your advice. If you have any comments, advice, recommendations or criticism about this article or how we could have handled things better, please let us know.

Thanks

Thank you to everyone who reported the issue. We would like to extend a special thanks to Lorenzo Colitti who reported the issue and also assisted us with trying new configurations, gathering information and general troubleshooting.