The Fascinating Story of ibiblio: From Their FTP Roots to Today’s Digital Repository of Public Information and Open-Source Software

blog post

By: Laura StameyPosted: November 29, 2016

Follow "The Hosting Blog" daily as hosting experts, programmers, and tech managers go in depth on a variety of tech-related topics.

TL; DR: As one of the first websites in the world, the predecessor of ibiblio set the stage for the ultimate “collection of collections.” An online public library supported by the University of North Carolina, ibiblio provides a range of publicly available information and programs spanning music, art, history, science, politics, and open-source software. The digitial archive is committed to public information and open-source communities, and the team behind it collaborates with some of the country’s most powerful hardware and software companies. Led by Director Paul Jones, ibiblio continues to preserve history while adapting to new technology standards and innovations.

Paul Jones is the definition of an early adopter. The University of North Carolina professor wrote email programs in the 1980s, before quitting email altogether in 2011. He started using Java in 1995, the year it was released. He hosted an FTP site before the World Wide Web existed and was among the first in the world to create a website.

That website combined his three passions and academic homes — computer science, journalism, and library sciences — and has since grown into ibiblio, a massive and diverse digital repository of public information and open-source software.

Hosting everything from Linux distributions and the world’s largest free text archive Project Gutenberg to etree’s live music sharing community and a database of slaves in Louisiana, ibiblio makes more than 1,500 resources available to the world.

“The journalism side thinks you should be able to find out about anything, and library sciences thinks you should be able to have access to it for self-improvement,” Paul said. “It’s not hard to say that if you have any of that in your background and you have the means to actualize ways to make that happen, that’s an opportunity and a gift that you should do.”

Archiving History: 3 Major Moments and Early ibiblio Accomplishments

Paul ran the internationally accessed FTP site for five years and watched network information discovery and retrieval tools gain popularity. He was interested in “helping connect astronomers with shared images, humanists with shared texts, all different kinds of things,” so Paul wrote a grant proposal to Sun Microsystems in 1992 to create web hosting for free information.

In addition to ibiblio, Paul Jones created one of the first websites in the world.

“Part of the deal with Sun was to find and recruit interesting content, and we did that by finding people who were being frankly kicked off their university or research servers because their stuff was too interesting to too many people, and they couldn’t deal with the capacity issues.”

1. Firsthand Account of the Soviet Union Collapse

When asked to put 24-plus years of ibiblio into perspective, Paul points to the multitude of big events that have played out on the site. As the Soviet Union began falling in the early ’90s, SunSITE turned into an unlikely source of information about a failed coup attempt.

“You couldn’t see it on television, but there was a Russian hacker sitting across the street typing in English about everything that was happening,” Paul said. “It was pretty incredible.”

2. Documenting Presidential Speeches and Materials From Clinton White House

The SunSITE team gathered and archived President Bill Clinton’s speeches before being chosen to help the White House with their first web initiatives to document his speeches, press briefings, and related materials during the presidency — ibiblio even archived the communications between Paul and the president-elect’s office.

3. WXYC Sends Radio Broadcasts Worldwide With First Internet Simulcast

Paul and SunSITE also helped the University of North Carolina’s student radio station, WXYC, become the first traditional radio station to broadcast on the Internet.

“We were moderately famous enough that we even became a Jeopardy! answer,” Paul said. “We invented streaming radio in the early ’90s, and they’re still doing it. Everybody does it now, though, so it’s a little bit different.”

Preserving and Protecting Resources: ibiblio Thinks Long Term

Through the decades, Paul said ibiblio is transforming from a library into more of a historical archive. “Originally, it was about getting a lot of new stuff in there, and now it’s about keeping useful things in a productive way that people can interact with in context,” he said.

Webpages from before 1999 aren’t much of a problem because they’re flat, static pages, according to Paul. The challenge comes with preserving database-driven, interactive sites built after 1999 such as a searchable database with information on people enslaved in Louisiana.

The database of Louisiana slaves is one of the resources Paul and the ibiblio team are trying to preserve.

“That database probably will not be updated or changed, but it uses a customized version of MySQL,” Paul said. “You want technology to get better every time you move forward, but each time you do, databases and complex websites are endangered.”

Preserving the functionality of older technologies is an issue that Paul said “hasn’t been paid the right attention to.” Some of ibiblio’s resources go stagnant when development money runs out, or the people maintaining the sites quit, retire, or die.

“You want to stop development on them and stop all the software environments and presets, but you want to keep it accessible and secure,” he said, adding that virtualization and containers might lead to a possible solution.

“Our largest challenge at the moment is to think long term about how to keep the collections available in context, not just as flattened webpages,” he said. “We want to keep them living, accessible, and secure, but not necessarily updated. That’s the philosophical challenge that reflects a technical problem.”

Innovating With Open-Source Software and Sponsors in Web’s Early Days

The ibiblio resources make up the largest collection of data leaving the university network. Even though the ibiblio servers receive 12 million to 16 million requests for information each day, the university’s large bandwidth needs mean that ibiblio doesn’t have to worry about scaling to meet traffic demands.

“As long as students watch Netflix, I’ll have plenty of bandwidth,” Paul said with a laugh. “They like bandwidth coming in, but I like it going out.”

Before ibiblio fame, the collection was known as SunSITE thanks to a grant from Sun Microsystems.

Paul and his teams have virtualized all of ibiblio’s machines and rely on university support for infrastructure support. Because of that assistance, Paul runs ibiblio with a very small staff — himself, one full-time employee, and two working students.

“You can do something like this without too much if you start early,” he said. “Even when we were working on White House stuff, I think we only had 12 people.”

Starting With Sun Microsystems: Grant Gets ibiblio Predecessor Going

A key component in the 1992 grant from Sun Microsystems that established SunSITE and ibiblio was to distribute free and open software, according to Paul. Drivers and device catalogs for SunOS, the precursor to the Solaris operating system, became among the first open-source software available on SunSITE. Sun, which sponsored the collection for eight years, eventually replicated SunSITE through 40 countries.

Central Place for Archiving, Documenting, and Distributing Linux

Around the same time, ibiblio began accepting and redistributing code bundles for their Linux archives. Popular distributions, such as Damn Small Linux and Puppy Linux, found an early home at ibiblio before the site stopped accepting new submissions in the mid-2000s. By then, the Linux collection comprised more than 5,000 files and had grown to 12GB.

“We have been one of the central places from which you can download Linux,” Paul said. “We were an early, early Linux proponent.”

Fostering Developer Communities and Growing With Red Hat

Paul fortunately connected with another budding open-source software company in the early ’90s, when they only had four employees. Now, Red Hat is a multinational open-source software company with more than 9,000 employees valued at more than $14.2 billion.

“Red Hat went from being edgy to being revolutionary, to being tried, trusted, and tested,” said Paul, who still has personal relationships with the four original employees. “They’re the only people who have successfully monetized Linux at that level. They manage to own a distro, brand it strongly, keep it open, and yet, make a profit and provide the right kinds of services and support. A lot of people have tried.”

When New Tech Becomes Mundane: ibiblio Evolves to Stay Ahead

After 25 years of watching the World Wide Web emerge, evolve, and explode, Paul still marvels at all the innovations he’s seen — as well as how quickly they become commonplace.

“The web was static at first, and then you suddenly had moving images, then streaming audio and streaming video,” he said. “The cool part is the part that was cool now seems so mundane.”

For example, he has watched a computer shrink from something that barely fit under his desk to a machine that fits in his hand and goes wherever he does.

Paul sees a similar transition happening with virtualization and container technology, which he hopes will greatly impact the preservation of ibiblio.

“I started working with this about four years ago, so I feel this is heading in the right direction,” he said. “I think we’ll have a mundane solution that everyone will think is normal. Successful technology is normal but not very flashy.”

Paul compares the evolution of technology to early cars, radios, and televisions — when they came out, everyone wanted to learn how to work on them. Eventually, though, you just want it to work. “You don’t care about the specs of your TV once it becomes mundane,” he said. “It should just work and fade into the background. Our goal in technology is to be mundane.”

Questions or Comments? Ask Laura!

Ask a question and Laura will respond to you. We strive to provide the best advice on the net and we are here to help you in any way we can.

Was this helpful? Tell Us Thanks.

Like this article on Facebook

Tweet this article on Twitter

Share this article on Google+

About the Author

Laura Stamey

Laura Stamey is a multi-talented communicator with a master's degree in web design and online communication and extensive experience creating web, print, and multimedia products. As a Contributing Editor for HostingAdvice.com, she combines her communication skills with web design experience, a passionate love for WordPress, and a strong desire to continue learning. When she's not addressing user questions via helpful how-to guides, you'll find her connecting with the folks behind companies that are powering the Web — covering their innovative technologies and the teams that engineer them.

Disclaimer: Great efforts are made to maintain reliable data on all offers presented. However, this data is provided without warranty. Users should always check the offer provider’s official website for current terms and details. Our site receives compensation from many of the offers listed on the site. Along with key review factors, this compensation may impact how and where products appear across the site (including, for example, the order in which they appear). Our site does not include the entire universe of available offers. Editorial opinions expressed on the site are strictly our own and are not provided, endorsed, or approved by advertisers.