[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Operational experience with 3 degrees
Bob,
Due to timing, I could not present the scheduled paper "Operational experience with Microsoft IPv6 based P2P application ("threedegrees") - Huitema, 10 mins". So I guess I will instead send a short abstract of my presentation in this message. Here it is.
On 2/26/03 we started the beta trial of "three degrees" (http://www.threedegrees.com <http://www.threedegrees.com/> ), a P2P IPv6 based application. ThreeDegrees allows users to organized in Small groups of peers, exchange IM, photos and winks, i.e. on screen animations, as well as jointly listen to music. There have been several thousand download of the beta.
The peer to peer exchanges are performed only over IPv6; in places where native connectivity is not available, the hosts use either 6to4 if they have a global IPv4 address, or Teredo if they don't. This deployment is in effect the first beta trial of the Teredo technology; Microsoft is providing a Teredo server for use by the Three Degrees hosts.
The short summary is that "it works", and that many of the early users are very happy. IPv6 is just "instantly deployed", validating the use of 6to4 and Teredo. The quality of the service, be it RTT or bandwidth, is adequate. But there are issues: some web servers became unreachable, some native IPv6 labs complained that they could not use the application, users behind symmetric NAT were left out, and there is a lack of Teredo relays.
After downloading and installing the beta, some users found themselves without access to a few of their favorite web sites; there were wild speculations that Microsoft had somehow disabled access to the sites. Well, Microsoft indeed did not do anything like that; the disabling of some web sites was a side effect of enabling IPv6. The version of Internet Explorer on Windows XP is IPv6 enabled. If IPv6 enabled on machine, IE will call "getaddrinfo" which in conformance with the IETF recommendation will look for AAAA record for the web site, and then look for an A record if there is no AAAA available. The problem is that, upon receiving a query for a AAAA record, some DNS servers returned a "no such domain" error, upon which getaddrinfo gave up. The issue affects about 20 of the top 500 web sites. We are glad that we found it during a limited size beta rather than during a wide scale deployment. We are working with the vendors of the affected DNS products and are confident that the bug will soon be fixed.
The second issue affected some native IPv6 networks, typically test laboratories in which a tester would load the three degrees application, and then complain that "it does not work in my IPv6 lab". The symptom was that the application will refuse to start, because it could not initialize the P2P software. This initialization requires that the host contacts the P2P "seed server", in order to obtain the address of a few other peers and connect to the peer-to-peer "cloud". The "seed server" is located at a 6to4 address, and is unreachable if UDP traffic is blocked by an IPv6 firewall, or in absence of a route to 2002:/16. The problem can be fixed by loosening firewall rules, and by making sure that the native IPv6 network has good connectivity with 6to4 nodes.
The third issue was not really a surprise. In our deployment, the only way for users located behind NAT to get IPv6 connectivity is to use Teredo, and Teredo only works through about 90% of the NAT available on the market; the others are "symmetric" NAT. Since users buying NATs have no clue whether their NAT is symmetric or not, this results in complaints such as "the application does not work behind a NAT of brand X". After the feedback from the beta, we better know the size of the problem, and we are making sure that several solutions will be available, to be used as appropriate. The Teredo software will be updated to automatically reserve a UDP port using the UPNP protocol when the NAT support this protocol, or to use a user assigned port when the user is able to open a port in the NAT using a management interface. If that fails, the user will in some cases be able to get a new firmware for the existing NAT, without having to buy a new NAT. Indeed, another solution would be to have generous third parties provide tunnel servers...
The fourth issue concerns the lack of Teredo relays. In our early deployment, Microsoft only runs a Teredo server, and each peer tries to run a "host specific" Teredo relay - a piece of software that intercept traffic bound to Teredo addresses and send it over UDP. However, a peer located behind symmetric NAT or firewall does not have enough IPv4 connectivity to properly run a Teredo relay, which results in connection failures in the P2P application. The solution is obviously to deploy a Teredo relay in the site exit routers of the peer sites, or possibly in the ISP networks. Teredo relays are even simpler to implement than Teredo clients or servers, and can incorporate a number of "stateful" traffic control features that make them resistant to attacks. To facilitate this deployment, we should provide a very short Teredo relay specification.
-- Christian Huitema