PSA: Don’t upload your important passwords to GitHub

The same goes for private SSH keys and other sensitive credentials.

It's akin to warning someone not to brush her teeth with a brick or to dry her hair with a blow torch, but based on numerous links circulating on Twitter Thursday morning, it bears saying: don't post sensitive account credentials to GitHub, or any other code repository.

On Thursday morning, the microblogging site was awash with messages linking to passwords and private cryptographic keys that are publicly accessible. Searches like this, this, and this turned up dozens of accounts that appeared to be exposing credentials that should never be made public. (Just minutes before Ars published this post, the searches stopped working, most likely as a result of GitHub admins who were trying to save users from their own carelessness. Many of the same GitHub accounts could still be located using Google, however.) Assuming they're still being used to log in to valid accounts, their exposure compromises the entire security that users attempted to establish when they generated the keys in the first place.

Ars won't be calling out individual accounts, although one GitHub offender appeared to reveal a password for an account on Chromium.org, the repository that stores the source code for Google's open-source browser. An eagle-eyed security researcher reported finding "an ssh password to a production server of a major, MAJOR website in China." Another tweet showed what appeared to be a sensitive GitHub authentication token used by a prominent front end developer for Bitly. In the wrong hands, a valid token could help miscreants redirect millions of people to malicious sites.

And it's not just GitHub users who appear to be loose-lipped, at least judging from links such as this one.

The practice of stashing sensitive log-in credentials in publicly accessible code repositories isn't exactly new, so it's unclear why it's only now receiving so much attention. It has touched off a major debate about whether GitHub bears any responsibility, with some arguing the fault lies solely with users and others saying GitHub has a duty to prevent misuse of its site.

Whatever view prevails, the tweets should come as a stern rebuke and a graphic demonstration that the practice is an extreme faux pas. Transgressors should reset or regenerate any compromised passwords or keys and purge the old ones right away. As in now.

Promoted Comments

Github has disabled search, presumably to buy time while they think of a way to address this. It's not github's *fault* that so many dot files and other sensitive files get uploaded to it, but I believe some simple heuristics could catch the majority of them and properly alert the repo owner. Ideally other code hosts would implement similar countermeasures.

It's really easy to see why so many people do this: most of these sensitive files are in hidden directories not visible by default in a file browser, and some tool the devs use use uploads visible and hidden files alike. They don't notice the list of committed files includes unintended ones and boom. Now I certainly spotted a few people who were intentionally backing up hidden configuration files, but it probably slipped their mind that ssh keys and the like were buried in there.

Not everything can be caught by these heuristics. One of the "best" finds I got today was via someone embedding an ssh username and password in an automated upload script. It belonged to a very major website in China. However, the SVN credentials of an open source developer on a project certainly worthy of targeting by malicious actors was a perfect example of unintentional commits that could easily be detected. For the record, those credentials were reported as compromised and revoked.

I have been in several arguments today about how much a service like github should coddle presumably technical users. If being a programmer meant you never made mistakes or oversights, well, I wouldn't have a job. If something has a sharp edge, sand it down as best you can. I love github and I would applaud some basic countermeasures to reduce the amount of private config files committed to online repos.

I cannot imagine any scenario where GitHub should be held responsible or to even have imagined such a significant number of careless users would do such a thing. This is 1000% on whomever checked in the code.

People always search for others to blame for their mistakes, ultimately they not responsible especially because developers should know better.

Hah!

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

Edit: Not that I'm really disagreeing with you, I agree they SHOULD know better.. I just know what the real world looks like too.

At least the machine in question doesn't answer ping. Its DNS resolves though, so it's not clear if it's just a Great Firewall issue, in which case the problem would be exploitable from inside the country but outside the corporate LAN.

I actually lost my copy of the ssh script, because I was looking at it in a mobile browser and then github search went down as I closed the page to go chat about it on twitter.

33 Reader Comments

I cannot imagine any scenario where GitHub should be held responsible or to even have imagined such a significant number of careless users would do such a thing. This is 1000% on whomever checked in the code.

I cannot imagine any scenario where GitHub should be held responsible or to even have imagined such a significant number of careless users would do such a thing. This is 1000% on whomever checked in the code.

People always search for others to blame for their mistakes, ultimately they not responsible especially because developers should know better.

I cannot imagine any scenario where GitHub should be held responsible or to even have imagined such a significant number of careless users would do such a thing. This is 1000% on whomever checked in the code.

People always search for others to blame for their mistakes, ultimately they not responsible especially because developers should know better.

Hah!

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

Edit: Not that I'm really disagreeing with you, I agree they SHOULD know better.. I just know what the real world looks like too.

Github has disabled search, presumably to buy time while they think of a way to address this. It's not github's *fault* that so many dot files and other sensitive files get uploaded to it, but I believe some simple heuristics could catch the majority of them and properly alert the repo owner. Ideally other code hosts would implement similar countermeasures.

It's really easy to see why so many people do this: most of these sensitive files are in hidden directories not visible by default in a file browser, and some tool the devs use use uploads visible and hidden files alike. They don't notice the list of committed files includes unintended ones and boom. Now I certainly spotted a few people who were intentionally backing up hidden configuration files, but it probably slipped their mind that ssh keys and the like were buried in there.

Not everything can be caught by these heuristics. One of the "best" finds I got today was via someone embedding an ssh username and password in an automated upload script. It belonged to a very major website in China. However, the SVN credentials of an open source developer on a project certainly worthy of targeting by malicious actors was a perfect example of unintentional commits that could easily be detected. For the record, those credentials were reported as compromised and revoked.

I have been in several arguments today about how much a service like github should coddle presumably technical users. If being a programmer meant you never made mistakes or oversights, well, I wouldn't have a job. If something has a sharp edge, sand it down as best you can. I love github and I would applaud some basic countermeasures to reduce the amount of private config files committed to online repos.

GitHub has absolutely zero obligation, and zero responsibility in the matter. The fault lies entirely with the developers that uploaded the keys in the first place. Entirely a PEBKAC issue.

Not to say that GitHub shouldn't take any steps, but anything they do would just be them trying to help people avoid mistakes, not because they must take action.

araemo wrote:

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

It really is quite shocking how many developers (and IT people!) know virtually zero about computers outside a very small domain of expertise. It seems rather difficult to comprehend how such a situation can arise... but it certainly does.

Are there seriously that many people who have their homedirs in git repos? Why would you do that?

You don't know about the dotfiles craze? Everyone wants to manage their dotfiles in git. For you windows people, those are hidden files that contain settings (kind of like the registry, but readable by both human and machine).

Aside: All this talk of FBI honeypot has got me thinking.. is that what prompted the creation of pedobear?

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

From blogs.msdn.com/b/ericlippert/archive/2007/10/11/packet-gnomes.aspx:

Quote:

The other morning I got the following question from a reader:

I have created a client-server application in C# using asynchronous socket methods [... blah blah blah, a long description of a scenario in which the socket misses packets under certain conditions]

My reply was that I believe networks are run by tiny "packet gnomes" who move the packets around on your behalf. Wireless networks are run by "packet fairies", who can fly. When I have to debug a network problem usually I either leave cookies out for the gnomes, or I call in a friend who has less magical/more scientific beliefs about networks.

At least the machine in question doesn't answer ping. Its DNS resolves though, so it's not clear if it's just a Great Firewall issue, in which case the problem would be exploitable from inside the country but outside the corporate LAN. Edit: oh, my (not so) old eyes, the DNS resolves to a 10.*.*.* (private block) so there's no way I could reach it... but... that shouldn't even be in public DNS records. Weird.

I actually lost my copy of the ssh script, because I was looking at it in a mobile browser and then github search went down as I closed the page to go chat about it on twitter.

It's akin to warning someone not to brush her teeth with a brick or to dry her hair with a blow torch

You'd think so, right?

A lady from accounting dept just walked into my office, looking to borrow a 'vpn cable'. Also, she was asking if the vpn cable was compatible with the accounting application we are running.

She's worked here for many years, every day all day at a computer, and using VPN from home.

Another time, a sales guy came in here in a huff, asking how to turn on a Dell tower. I had the same model at my desk, and rather than calling him a stupid moron in his face, I pointed at the one and only button on the entire front of the computer, the one with an incomplete circle with a line through it and it's glowing amber. He said, again in a huff, "oh so that's what I need to find" and stomped off. He too has worked here for several years, on a computer all day every day.

I don't think my users are any dumber than anyone else's.

They usually understand hardware problems. If you drop it, or spill on it, or it falls into a toilet, it is broke.

But software issues? Revealing sensitive computer info? What's that?! Why would that hurt anything? Why would they care? Trojans? Too many startup items killing the computer? They can't understand those concepts. They just can't.

I cannot imagine any scenario where GitHub should be held responsible or to even have imagined such a significant number of careless users would do such a thing. This is 1000% on whomever checked in the code.

People always search for others to blame for their mistakes, ultimately they not responsible especially because developers should know better.

Of course the developers are responsible for those mistakes, but it ridiculous to ever use the statement "should know better" when talking about such a easy to overlook issue when dealing with the often extremely complicated projects. What a person "knows" rarely if ever comes into play when dealing with the 100s of things that need to be checked for a commit, and including configuration checks for possible security conflicts can easily be missed in the shuffle.

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

From blogs.msdn.com/b/ericlippert/archive/2007/10/11/packet-gnomes.aspx:

Quote:

The other morning I got the following question from a reader:

I have created a client-server application in C# using asynchronous socket methods [... blah blah blah, a long description of a scenario in which the socket misses packets under certain conditions]

My reply was that I believe networks are run by tiny "packet gnomes" who move the packets around on your behalf. Wireless networks are run by "packet fairies", who can fly. When I have to debug a network problem usually I either leave cookies out for the gnomes, or I call in a friend who has less magical/more scientific beliefs about networks.

Clearly the author is being facetious. Personally, I have to disagree that this is an example of a computer-illiterate developer. Most of us have specializations of some kind, and we can't know everything about every technology. We don't get to hear the specific conditions which the question pertained to, but as a network engineer I can say that diagnosing intermittent packet loss isn't trivial. If Eric is a C# guru specializing in code security, I wouldn't necessarily expect him to run up and down the TCP/IP stack troubleshooting packet loss. This is just like you wouldn't expect me to do code analysis to identify ASLR vulnerabilities, since I do networks. This doesn't make anyone deficient in their skills, it just means that the question was asked to the wrong person.

As far as submitting dotfiles and ssh keys to public repositories? Totally irresponsible and the developers should have been paying more attention, but there's nothing wrong with GitHub providing a safety net to help prevent this kind of thing from happening.

Sooner or later everyone who does anything fucks some seriously simple things up totally. If you work with computers or software, you gonna fuck that up. It's not if, it's when.That's not to say you're not responsible then. But everytime I laugh about such things and point a finger, I know sooner or later I will very likely be the one on the other end of the finger.

So yeah, it's not Github's job to make knives childsafe. But if they decide to add a warning sticker that would not be a bad thing either. If people wouldn't make mistakes that are technically unnecessary, liability insurance would not have saved countless peoples' asses.

One thing to remember is that you definitely need to be storing these credentials in a versioning repository somewhere within your organization. The notion that operations doesn't need version control is idiotic. Not as idiotic as uploading your unencrypted private key to GitHub, but don't use this as an example to have your sysadmins store credentials outside of SCM. They belong in source control too. Note that I don't mean their personal private keys, I mean credentials that are needed to deploy and configure code to staging and production systems. You should have separate repos for application code that your developers need access to and application configuration that your ops team needs to do their job.

And not to put too fine a point on it, all your organizations pubkeys should be in LDAP or equivalent and all your servers should check LDAP to grant access. In the unix world, you can even write a custom shell that acts a gatekeeper after login to sandbox users depending on what access is allowed in LDAP.

It's really easy to see why so many people do this: most of these sensitive files are in hidden directories not visible by default in a file browser, and some tool the devs use use uploads visible and hidden files alike. They don't

But there is still something weird about pushing dotfiles (other than .gitignore) up to github, especially sensitive ones. OK, people will accidentally push up a .gdb or a .foobar.py.swp, but nobody has SSH keys in those files.

abadidea wrote:

notice the list of committed files includes unintended ones and boom. Now I certainly spotted a few people who were intentionally backing up hidden configuration files, but it probably slipped their mind that ssh keys and the like were buried in there.

Now here's were stupidity-for-the-facebook-generation comes in. This is people intentionally publishing personal info onto the web, so *of course* thing go wrong. I myself keep dotfiles under revision control (although id don't allow SSH private keys into the repo), but even if I did, I certainly don't push it up to the web.

I myself keep dotfiles under revision control (although id don't allow SSH private keys into the repo), but even if I did, I certainly don't push it up to the web.

It -almost- actually happened to me sometimes. Keep in mind that it is not necessarily "hidden" (dot) files where you might store sensitive credentials. AFAICR in my case it was some login/scratch/toy/playground script where I stored my DB credentials (for testing reasons). Or I even hardcoded "for a short amount of time" the credentials into a source file "for testing purposes only" (because I was annoyed at entering the credentails each time upon program launch - you have no idea how lazy a developer sometimes becomes

Off course that was a "hobby project" of mine and the credentials were only useful for my local MySQL DB instance. I had already committed to my local repository and just before pushing my changes to GitHub alarm bells rang! "Oops, what about that password again?"

Luckily I realised it *before* I pushed it into the public repository and could "change the past" (can't remember the exact Git command, but unlike with e.g. SVN you CAN modify changes committed in the past with Git, so I could remove my password traces in the change history).

The point is you would be surprised where a developer might want to store "temporarily" credentials for a) integration servers b) code signature servers c) FTP accounts d) ... purely out of convenience, you have changed several dozens of files in one commit, "add all", and *bang* there you go, your "temporary" login script gets added/committed as well!

That's not because "you don't know". It can simply happen because "Puh! Done with that change! THAT was a hell lot of work! Gotta write that commit comment still and then I'm done... *flush and commit*"

Now, whenever you upload anything to the "cloud", just replace that with "Somebody Else's Computer." Are you as comfortable doing it?

The real truth is that "cloud" is not just somebody else's computer, but it is almost always "some company trying to make money with my data that doesn't care about my privacy and will share it all with the world for $0.000005".

Do you really think that some tick, "private repo", inside a shared DB is enough protection for private data?

Now, whenever you upload anything to the "cloud", just replace that with "Somebody Else's Computer." Are you as comfortable doing it?

Not sure what your point here is really, but the code you upload to GitHub is meant to be shared with other people! So we talk about people who accidentally shared information they shouldn't have uploaded in the first place.

Except off course if you are using a private Git repository (which costs money, in contrast to the public ones). But then you were also not affected by that "public search" functionality either and could upload whatever sensitive data you wanted (given all the trust you have in the security of the GitHub servers). But again, that is not what we are discussing here...

I store my dotfiles in an encrypted 7z archive and upload it to Dropbox. I've always felt a bit uneasy about doing so, even though the archive should be fairly safe (7z uses AES-256 and my password is long). It beggars belief that people upload their dot file to *publicly* available source code repositories, in plain text.

The number of 'developers' I've known who are completely computer-illiterate outside of their specialty is silly. Yes, you'd think it'd be impossible to be a successful developer without a decent amount of computer literacy, but.. no.

I have to agree 100%. I have been programming for over 30 years and worked in IT going on 20 years. Most developers/programmers I have known are the least technically knowledgeable people I have ever met. Even the simple things never even cross there mind because it is not directly within the realm they work. So they wonder why something works on one machine, but not another, without ever realizing that one machine ahs Java 7 while the other still has Java 3.