Recently in Penn State Category

Oy, it's been a while since my last update. Lots of news to report. I'll spread that out over several posts.

Last month, I gave a talk on Campus IPv6 Deployment at the Winter 2009 Internet2 Joint Techs workshop in College Station, TX. The gist of the talk was that the main barrier to IPv6 deployment is attitude and perception. Although there are plenty of technical issues to overcome, the technology is good enough for initial deployment, and we have to start doing that now. The hurdle to overcome is attitudes like "no one is using IPv6" and "IPv6 is a network issue."

People are using IPv6. And there's a lot more to a successful IPv6 deployment than upgrading some routers and firewalls.

I've never seen IPv6 as a network issue. Maybe it's because I'm not a network engineer. Maybe it's because I've always thought the OSI model was a fiction. But I approach IPv6 deployment from the perspective of services not devices. E.g., what services do we need to IPv6-enable to make IPv6-only machines viable from a policy compliance perspective?

We have some good news to report on this front. At Penn State, we've got a fair bit of buy-in about the importance of IPv6 from the senior leadership in central IT. Our deputy CIO has tasked me with leading a team to develop an IPv6 deployment plan for central IT. We're only about two months in, so we don't have much to report yet. I should say that we are only charged with developing a plan; we're not yet implementing it, although I am pushing for little bits of deployment whenever I can. We should have the final report done in a few months, and I should be able to post more then.

Educause has allowed subdomains to register IPv6 glue for a while (psu.edu has had IPv6 glue for at least year). The DNS root got IPv6 support earlier this year. But .edu was always this big v4-only gap in DNS. I'm really happy that it's being fixed.

;; ADDITIONAL SECTION:
A.GTLD-SERVERS.NET. 78012 IN A 192.5.6.30A.GTLD-SERVERS.NET. 172700 IN AAAA 2001:503:a83e::2:30
C.GTLD-SERVERS.NET. 78008 IN A 192.26.92.30
D.GTLD-SERVERS.NET. 78008 IN A 192.31.80.30
E.GTLD-SERVERS.NET. 78008 IN A 192.12.94.30
G3.NSTLD.COM. 2887 IN A 192.42.93.32
H3.NSTLD.COM. 2887 IN A 192.54.112.32
L3.NSTLD.COM. 2887 IN A 192.41.162.32
M3.NSTLD.COM. 172700 IN A 192.55.83.32

I mentioned recently that I gave an IPv6 poster session at the Penn State WebConference. I've put the poster and handout online. I spoke to a couple of interesting people at the conference, and learned a lot about the web infrastructure at the University (there are more IIS and ColdFusion shops that I realized. Fortunately, both supports IPv6.)

In DNS news, the .dk (Denmark) and .nk (Saint Kitts and Nevis) top-level-domains are now reachable via IPv6. This brings the total to 190 of 269 TLDs that are reachable over IPv6.

Penn State is a multi-campus institution. The University Park campus is commonly thought of as the "main campus," but we have 26 other commonwealth campuses. We also have an affiliate relationship with the Pennsylvania College of Technology. I have a personal connection to Penn College -- both my mother and step father worked there for over a decade, and both were deans there for several years.

Penn College isn't part of Penn State. ITS does not provide networking services for them, and Penn College has its own IP address space. I was trolling around whois last week and noticed that Penn College has IPv6 space! And they're using it: The Penn College chapter of the ACM has IPv6 on their web site.

In my previous blog entry, I reported at PSU has about 300,000 IPv4 addresses assigned to it, and that ITS manages a little more than half of those. ITS projects that it will exhaust its IPv4 pool sometime in late 2009 or early 2010. This graph is based on data from TNS in January 2008:

The text on the graph is hard to read. Click on it for a larger version.

When we finally exhaust our pool, will we able to go ARIN
and get more? That depends on the status of ARIN's IPv4 pool and on any
new policy ARIN has adopted. Several of the RIRs have adopted policies
which make it much more difficult to obtain additional IPv4 space. I
believe that ARIN will still have addresses left, but we probably won't
be able to get space due to policy changes. If that's the case, it's a
pretty scary realization. Quite likely, we'll be stuck with the IPv4
space we already have.

It will be several years before ITS runs out of IPv4 space, but the end is closer than many people think. No, the sky is not falling, but I firmly believe that we have to start planning now for what to do when we do run out of space. IP addressing will be a serious issue during the University's next strategic planning period (2009-2014).

Several network managers on campus have already begun efforts to conserve IPv4 space. They're doing things like moving networked printers and internal servers to private address space. But there are only so many printers to move! These steps are important in the short-term to preserve business continuity, but ultimately, they only relieve the pressure temporarily. We're still left with the question: What is our long-term solution to the address crunch?

Some within the university have proposed using static NAT for desktops. This approach has issues, including potentially breaking certain applications. The number of apps broken by NAT has declined in recent years, in part due to protocols such as STUN, TURN and ICE, to name just a few. Frequently, NAT-enabled apps will have to support multiple such protocols. Adding NAT traversal support to an application can add considerable development and testing costs (see this paper on Skype for a good example of NAT-enabling a real-world app). Given the aggressive use of NAT today, many developers are already adding NAT support to their apps.

Others are proposing using IPv6. Of course, deploying IPv6 isn't free either. There are both hardware and software costs involved. New routers and firewalls have to be purchased. Of course, network equipment needs to be regularly replaced anyway to keep up with increased user demands. If IPv6 support is introduced as network equipment is replaced, deployment costs can be considerably reduced (but not eliminated). Software may need to be upgraded to use IPv6-safe APIs or to remove hard-coded IPv4-assumptions. There will certainly be testing costs. Fortunately, many programming languages (Python, Ruby, Java) and networking libraries (CFNetwork, Qt, NSPR, APR) are already IPv6-enabled. This support tremendously helps reduce programming costs. Even so, there will certainly be at least a few legacy apps which will be very expensive to convert to IPv6.

Ultimately, I think this is an economic question: In the long-term is it cheaper to (more aggressively) deploy NAT and NAT-enable our apps, or to deploy IPv6 and v6-enable our apps? Any measurement of cost should include the factors outlined above plus opportunity cost.

My vote is for IPv6. Why? Because the graph above scares me. I'm just not comfortable living in a network where we've used up most of our publicly routable addresses. Even with extensive use of NAT, I'm afraid that such an environment limits our ability to deploy new applications and services. Even if Penn State can reclaim significant portions of its IPv4 space, the
story doesn't end at our border router. What about our partners? What
about network operators in Europe and Asia who are already deploying IPv6? If we need to interoperate with them, we'll need an IPv6 story of our own.

A little over half of those addresses are managed by ITS. While it seems like PSU has a lot of addresses (and we do), ITS estimates that we'll run out in less than 5 years. More on that in a future blog entry.

If you are IPv6-enabling a website protected by WebAccess, you might need to make a change to your web server's configuration. Specifically, you might need to set CosignCheckIP never in your Apache config. Otherwise, connections over IPv6 won't be authenticated. You can read more about why this happens in the ITS wiki.

Penn State's WebAccess product is based on CoSign, an open-source web single-sign-on system. The CoSign source code is not IPv6-clean, so even if ASET were to assign IPv6 addresses to the WebAccess servers, we'd still have problems. I'm working on a patch to CoSign to make it IPv6-clean. After the break, I'll do some more testing and submit it to the CoSign project for integration.

I had some freetime this afternoon, so I whipped up a script to check IPv6 deployment at the Big Ten. I checked for AAAA records for authoritative nameservers and SMTP servers.

Only University of Illinois, Urbana-Champagne and Penn State had any IPv6 presence. We both have one IPv6-accessible authoritative nameserver.

Yes, otc2.psu.edu is IPv6-accessible. It's answering DNS and NTP queries over IPv6. At the moment, it's not answering recursive queries over IPv6, but TNS is aware of this issue and hopes to have it corrected soon.