Guest Blog: Globus and the path to sustainability

It’s been more than twenty years since the first public release of Globus software. In that time, we’ve tried several routes to sustainability and learned a few things we’ll share in this post.

The core mission of Globus—which has changed very little since 1995—is to increase the efficiency and effectiveness of researchers engaged in data-driven science and scholarship through sustainable software. This is what we’re trying to sustain, and there are a few important points to notice. First, we’re focused 100% on researcher productivity. We measure Globus’s success exclusively by how we’ve helped researchers. Second, we believe developing and operating production grade software are our group’s strengths and are how we’ll achieve our mission. Finally, we feel strongly that “sustainable” doesn’t only mean “continuously available.” It also means continuous updates, adaptations for changing technologies, responsive user support, and enthusiastic users. This mission has guided and focused our efforts toward sustainability.

What we’ve tried

Our first public product was the Globus Toolkit: open source software for enabling distributed computing. It provided services for resource providers (organizations offering high-end computing systems to researchers) to install on their systems to provide a common platform, and clients for research application developers to access the platform services. We developed and supported the Globus Toolkit for fifteen years, trying out several sustainability strategies. The initial releases in 1998 and 1999 were open source, and from the beginning there were contributors who helped make it useful to researchers. In 2003, we formed an international research consortium, the Globus Alliance [1], hoping to encourage broader research and academic support. In 2004, several of us launched a company, Univa [2], hoping to raise money by offering support services for enterprises. In 2005, we formed an industry organization, the Globus Consortium [3], with members including IBM, Sun, Hewlett-Packard, and Intel. [4]

In 2009, still feeling pressure to sustain our mission, we pivoted to a new strategy with three key elements: a tighter focus on data management, a “freemium” support model [5], and cloud-based Software-as-a-Service (SaaS) delivery. [6] We launched the new services in 2010 and signed our first premium subscription in 2013. We made our last substantial contribution to the open source Globus Toolkit in 2014 with version 6.0. Earlier this year we signed our 100th subscriber under the new model, and an increasing portion of our ongoing funding has shifted from federal grants to institutional subscriptions.

Our evolution is ongoing, of course, but let’s take a look at what we’ve learned so far.

Open source: an important but incomplete solution

The first lesson we learned was that FOSS (Free and Open Source) is neither sufficient nor strictly necessary to sustain research software. We initially thought software had to be FOSS for researchers and resource providers to use, a common perception/expectation within the research and education community, but one that is often at odds with building sustainable software. We’d also hoped other research developers—while working on their own problems—would add features they needed to the software and help support it, eventually freeing us to work on new problems. After a decade of trying, we realized it wasn’t working as we’d planned.

As we worked with researchers and their support staff, we learned there are very few software developers—far fewer than we’d hoped—with the freedom to focus on research productivity at large. [7] While there had always been contributors from other organizations, there weren’t enough to sustain things without our team’s constant contributions. Contributors were seldom able to provide ongoing maintenance or user support for code they contributed, much less for the rest of the toolkit. But maintenance and user support are crucial for software to be useful.

Thinking about how to reconcile the preference for FOSS with the reality that it would not work for the Globus mission, we identified three key elements of FOSS’s value to researchers. First, researchers would like the ability to independently inspect and validate the code for security and privacy issues. Second, they need to be able to continue using the code if the original developers abandon it. Third, they want to be able to “fork” the code, making and maintaining a separate copy, often so they can add proprietary (private) features for their own use. The first and second points are clearly beneficial, and both are possible with traditional licensing. [8] The third point seems useful in the short term, but it also speeds the software toward an early death. As copies (“forks”) are maintained by separate teams, new features are no longer shared with the community and the developer community is increasingly divided. [9] This works against the sustainability of FOSS projects.

To summarize: the sparsity of contributors, the availability of benefits by other means, and the need to avoid “forks” led us to conclude that—while it serves a valuable purpose—FOSS alone wouldn’t sustain Globus.

Focus on researchers

We pivoted to Software-as-a-Service (SaaS) in 2008 and a freemium model in 2013 when we realized our approach to our mission was unnecessarily indirect. As described earlier, we designed the Globus Toolkit with two primary users in mind: research application developers and resource providers. (Both groups support researchers, but neither are, necessarily, researchers themselves.) The Globus mission is to improve research productivity, so instead of designing software for people who support researchers, we should design Globus for researchers.

Researchers need applications with 24/7 availability, little or no installation or configuration, and simple, intuitive interfaces. [10] A software toolkit doesn’t provide those features, but SaaS does. SaaS delivery means our team offers hosted web applications (web apps) for researchers who access the applications in their web browsers. This frees researchers from installing and configuring the software. It also gives our team control of (and responsibility for) the end-to-end user experience.

Users and customers

Realizing that our primary users would be researchers complicated our sustainability strategy. First, there are a lot of researchers: far too many for us to sell anything to individually. Second, the funds that researchers have access to are from grants, which are highly competitive, so little is available for services like ours. This argued strongly for a free service for researchers, but it left unanswered the question of how we would support our work.

Basic research services—local computing, support personnel, administrative services—are most often provided by the institutions where researchers work. Computing is spearheaded by a local (often research-focused) computing organization. There are a manageable number of these, charged with acquiring and providing leading-edge services to their researchers. Leading research institutions view computing as a strategic investment, so that’s where we could make an argument to support our work. Our premium subscriptions are consequently designed to offer value to research computing directors and, by extension, to the research community they serve.

What do research computing directors need that a Globus subscription can provide? Perhaps most importantly, they need a fully-operational, managed service that keeps their researchers productive and that they don’t have to run themselves. Beyond that, a Globus subscription meets needs they’ve been trying to satisfy on their own for years.

It makes their researchers’ data wrangling easier.

It reduces friction when collaborating with other institutions.

It offers ways to automate research data flows.

It allows the research computing organization to use--or try out--new types of storage while giving researchers a unified interface.

It provides visibility into research storage utilization.

It adds robust (and secure) data management to research applications.

It optimizes data transfer performance for researchers.

It provides access to expert support resources.

These are benefits research computing organizations recognize and are already budgeting for.

Investing in the platform

Although our core products are now aimed at researchers, our mission still drives us to encourage and support research application and service developers: a growing community that’s clearly important to the future of research. The research community is characterized by many small teams with specialized needs that require custom applications. Research application developers—in scarce supply but poised to dramatically improve research productivity—need a development platform that enables large-scale and automated data management.

Our SaaS applications are built on an emerging Platform-as-a-Service (PaaS) [11]: a collection of services with REST APIs [12], designed for research application developers, and accessible via a software development kit (SDK) and a command-line interface (CLI). We currently use the Globus platform ourselves for building and operating our primary SaaS product for researchers, and plan to invest substantially in expanding its functionality so that developers can build richer solutions for their research communities.

Conclusions

We’ve tried many ways to allow Globus to live and grow over twenty years and we’ve made good progress. To get to this point, we’ve had to think carefully about what we need to sustain and we’ve had to find a strategy that matches our mission. We’ve learned that while FOSS has helpful features, it’s not a strategy for sustainability. We’ve also learned that our users and our supporters can be different people. For that to work, our primary product has to focus on our users and our revenue mechanisms have to focus on our supporters. Our current sustainability model—characterized by SaaS, free use for researchers, and premium subscriptions for institutions—has energized and excited our team, and it’s been enthusiastically received by the research community. We continue to evolve the model as we learn more from our subscribers, and we look forward to reaching sustainability and having the opportunity to continue serving the community we’re a part of.

[7] Incentives for commercial software developers to contribute to FOSS are complicated. Research developers are scarce and needed for urgent problems faced by their research teams. Even when development teams have the freedom to work on forward-looking FOSS software—such as the members of our research consortium, the Globus Alliance—it’s challenging to identify and forge a coherent vision of what’s needed and worth building.

More than just everyday musings...

This project is funded by the National Science Foundation under award number ACI-1547611. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.