Qualcomm & GitHub

First, 3G allowed us to text, send emails, and use GPS from our phones. Then, 4G supported immersive gaming, life-changing apps, and photo-based social media. Now, 5G is teetering on the horizon: A transformative generation of connectivity that will once again evolve the way we connect with each other and the world. A driving force behind this innovation is Qualcomm. Founded in 1985 and headquartered in San Diego with over 220 worldwide locations, 35,400 employees, and more than 130,000 patents, the company has taken great strides to lead the mobile industry forward. And it’s not stopping any time soon.

Qualcomm Technologies, Inc. operates Qualcomm’s engineering, research and development functions. Within Qualcomm Technologies, there is a broad range of employees supporting functions and tools that allow the company to run effectively across a multitude of platforms and countries. This means there’s a lot of shared code, as well as IT and supporting business functions that use version control to keep developers moving quickly, store policies, stay compliant, track records, and more.

Director of Engineering Craig Northway explained that prior to GitHub, Qualcomm engineering’s rigid version control system caused a lot of friction. “Deploying GitHub in 2011 enabled a self-service option for people to quickly get to work. Side by side, other tools weren’t as complete in terms of operational complexity.” Today, Systems Engineer Greg Golin is the sole administrator, dedicating only part of his time to maintenance. This is compared to older systems, “which required a fair bit of hand holding, even for a small set of teams,” said Golin.

Over time, GitHub usage grew and adoption happened organically within Qualcomm Technologies. Northway explained that “the biggest thing we see is a reduction in developer friction. Keeping developers happy and productive is super valuable. It’s just a couple clicks, and they’re up and running.”

Of Qualcomm Technologies’ 6,500 total GitHub users, 76% are engineers (35% in software, 8.5% in systems, 8.5% in ASICs, 7.5% in hardware, 6.5% in IT, 5% in program analysis, and 5% in system testing), with 24% from various other departments. Senior Staff IT Engineer Paul Krizak said that “within each of these classifications there’s obviously some blending.” On his team, they store not only source code but also their configuration management policy. Because the policy is made up of text files that look like source code, it’s also stored in repositories. When changes are made, they’re pushed out through a webhook so teams can seamlessly make and track changes, get approvals, and deploy.

Krizak said his teams also use GitHub in seemingly unconventional ways. “We make very heavy use of README files. It may sound backwards, but each repository represents a discrete piece of policy and allows us to have a very well-formatted piece of documentation embedded in the code.”

While many people associate GitHub with source code, Krizak knows it’s more versatile. “I think of it as a document of record repository. As long as it’s text, you can put it on GitHub. Even if it doesn’t change often, you reap huge benefits by storing information there.”

Being able to have access control and security built around the data is another advantage. “You can’t do that with a file system or database. And you don’t get a history of changes. When you treat GitHub as a universal tool, it creates a lot of opportunities and introduces interesting new workflows to the process.” Golin added that in systems engineering, “it gives us another avenue to get work done without involving a bunch of other people or opening tickets and waiting.”

From the packaging team, Project Manager Tarun Sadhnani appreciates the ability to generate discussions, make comments, and access the history of issues. Even if you close out an issue or it’s archived, the link is still active, which grants historical knowledge and access to anyone who needs it.

In addition to smoother processes, teams avoid extra work and redundancies. One of Northway’s database systems stores software and product composition information accessed by numerous teams. “When you write a Python binding to the database and store it on GitHub, everyone can find it. If organizations are restructured or people leave, no code is lost. We don’t have to rewrite it or spend money doing something that’s already been done.”

Krizak explained that on the configuration management side, they have small units of configuration policy called “duties”. Most duties had no version control—or at most an RCS directory that would contain some history if admins remembered to use it. As the number of engineers grew, so did the configuration management system. Managing thousands of policy files became an unsustainable process. In addition, there were an increasing number of duties that controlled compliance, passwords, elevated permissions, and more. These duties carry audit and governance requirements that RCS alone was incapable of providing.

“Before GitHub, the configuration management policy was kind of a dark art,” Krizak noted. When someone needed a change, they’d send an email or raise a ticket. “Code changes were relegated to a handful of people in an ivory tower.” With everything in GitHub, employees can make changes and open pull requests themselves. Krizak’s team simply has to review and merge. “It’s made what we do much more seamless, transparent, and collaborative.”

Now, the team works with a new system where duties can be “mastered” in GitHub repositories, using a webhook to refresh the global filesystem when changes are pushed to the master branch. A script assists end-users with transitioning their duties to this new system. “The script uses the GitHub API to create the repository, sync the data, and set up all the correct webhooks, branches, and triggers,” explained Krizak. “It ensures the duty is always correctly mapped to the right repository.” A user simply types in a command, then enters their username and password. The script and API do the rest.

Before 2011, there were incongruous ways of storing things for Qualcomm engineers. This meant that when compliance or security was involved, it was a massive ask for Krizak’s team. With repositories, “GitHub creates a predictable pattern for achieving compliance. We keep finding new places where compliance or security needs a record. And we’re able to provide that by making GitHub the canonical source.”

On the packaging side, each product has a ticket, which becomes its single source of truth. Sadhnani explained that, “Whether it’s being initiated, planned, executed, or terminated, the ticket contains all relevant information, including its owner.” They use a template to maintain consistency and stickers to generate a weekly report with the GitHub API. “There’s less manual work, and leads can send stickers out to the wider teams,” said Sadhnani.

Though many employees in larger companies like Qualcomm Technologies may not be as familiar with GitHub, new users are quick to realize its value. Training is available, and “the benefits make themselves apparent very quickly,” said Krizak. Take a rollback, for example. What was once a painstaking process is now practically effortless. “It’s just one click. Revert! You’re done.”

GitHub has also allowed Qualcomm Technologies to work better together across multiple time zones, from San Diego and India to Cambridge, Bangalore, Shanghai, and Austin. Krizak said, “It’s essentially impossible to have face-to-face meetings, but through pull requests and comment chains, we can work as a cohesive team. GitHub is a key component in making that 12-hour time zone difference usable.” Added Northway, “Using GitHub and tying that into JIRA to hand off tickets and reviews is vital.”

Between cross-continent collaboration, historical cataloguing, and streamlined workflows, GitHub has revolutionized the way teams work at Qualcomm Technologies. As of late December 2018, the company had 37,000 repositories and 10,000 organizations, with nearly 600 commits and 900 pull requests merged daily. When they do need help from GitHub, Golin said, “The support is stellar. Consistently great.” In the end, GitHub became integral to business in a way the teams didn’t initially expect. Krizak explained: “We were able to create a culture around using GitHub as a tool, not just a place to store code.”

Product

GitHub Enterprise Server

Repositories

37,000+

Number of seats

6,500

Customer since

1/14/13

Industry

Telecommunications and Semiconductors

Bring GitHub to work

From flexible hosting to data‐powered security,
get everything your team needs to build at their best.