Gem dependency use among Sufia/Hyrax apps

jrochkind

7 months ago

Advertisements

I have a little side project that uses the GitHub API (and a little bit of rubygems API) to analyze what gem dependencies and versions (from among a list of ‘interesting’ ones) are being used in a list of open Github repos with `Gemfile.lock`s, that I wrote out of curiosity regarding sufia/hyrax apps. I think it could turn into a useful tool for any ruby open source community using common dependencies to use to see what the community is up to.

It’s far from done, it just generates an ASCII report, and is missing many features I’d like. There are things I’m curious about that it doesn’t report on yet, like history of dependency use, how often do people upgrade a given dependency. And I’d like an interactive HTML interface that lets you slice and dice the data a bit (of people using a given gem, how many are also using another gem, etc). And then maybe set it up so it’s on the public web and regularly updates itself.

But it’s been a couple of months since I’ve worked on it, and I thought just the current snapshot in limited ASCII report format was useful enough that I should share a report.

The report, intentionally, for now, does not tell you which repos are using which dependencies, it just gives aggregate descriptive statistics. (Although you could of course manually find that out from their open Gemfile.locks). I wanted to avoid seeming to ‘call out’ anyone for using old versions or whatever. Although it would be useful to know, so you can, say, get in touch with people using the same things or same versions as you, I wanted to get some community feedback first. Thoughts on if it should?

I got the list of repos from various public lists of sufia or hyrax repos. Some things on the lists didn’t actually have open github repos at that address anymore — or had an open repo, but without a Gemfile.lock! Can only analyze with a Gemfile.lock in the repo. But I don’t really know which of these repos are in production, and which might be not yet, no longer, or never were. If you have a repo you’d like me to add or remove from the list, let me know! Also any other things you might want the report to include or questions you might want to let it help you answer. Or additional ‘interesting’ gems you’d like included in the report?

I do think it’s pretty cool that the combination of machine-readable Gemfile.lock and the GitHub API lets us do some pretty cool stuff here! If I get around to writing an interactive HTML interface, I’m thinking of trying to do it all in static file Javascript. That would require rewriting some of the analysis tools I’ve already written in ruby, in JS, but might be a good project to experiment with, say, vue.js. I don’t have much fancy new-gen JS experience, and this is a nice isolated thing for trying it out.

I am not sure what to read into these results. They aren’t necessarily good or bad, they just are a statement of what things are, which I think is interesting and useful in itself, and helps us plan and coordinate. I do think it’s worth recognizing that when developers in the community are on old major versions of shared dependencies, it increases the cost for them to contribute back upstream, makes it harder to do as part of “scratching their own itch”, and probably decreases such contributions. I also found it interesting how many repos use unreleased straight-from-github versions of some dependencies (17 of 28 do at least once), as well as the handful of gems that are fairly widely used in production but still don’t have a 1.0 release.