Tor Development Reports

Hi all. Changes at work have me stressed so I’ll be skipping Rome this year, but none the less Tor has been a welcome anchor. Work may suck, but Tor? Well…

ORPort Protocol Support

As discussed yesterday Stem can now communicate over the ORPort protocol. Still lots of follow-up work to do, but thanks to Tim’s wonderful work prototyping how this is done with Endosome Stem can now download descriptors via the ORPort protocol!

Packaging Community

Commonly major releases are followed by followup packaging work and Nyx’s recent release was no exception. But rather than simply work with our delightful packaging community as I usually do I decided instead to bring order to the chaos.

How did you first get involved with the Tor Project, and with internet freedom activism more generally?

This application was not accepted. Tor took on several students, but unfortunately their top two picks (me and Runa) applied for the same task so they couldn’t accept us both. In the end though I’m glad for it since that’s how I first got involved authoring Nyx.

What are the main things you’ve worked on with the Tor Project? Which of these are you the most proud of? What are you working on at the moment, and what does an average day look like for you?

In short my main involvement with Tor is Nyx (CLI monitor for tor relays) and Stem (Python library for Tor). I ran Tor’s GSoC program for six years but recently I turned in that hat, and now act as our membership secretary (onboarding new folks, maintaining parts of the site, etc).

As for my average day, generally I do Tor stuff for a couple hours as I sip my morning coffee before work, and take on larger projects over the weekend. My day job is as a software engineer at Amazon.

What would you say are your main motivations for the work you do?

Tor is interesting in that there’s a wide range of interests in our community. For some its privacy, others counter-censorship or freedom of speech. For me my foremost interest is in Tor as an open source community.

Do you see yourself as part of a Tor community? Do you think there is a strong community around Tor, and what are the main ways that the community interact with one another?

Yup to both. Most common forms of communication are email, irc, and developer meetings.

Do you think people in the Tor community are quite similar, or are there a lot of different views and perspectives?

Good question. I’d say that our community is uniform in its libertarian bend. Tor is a privacy and counter-censorship tool after all, so everyone in our community tends to believe in privacy and free speech. That said, this still leaves quite a bit of room for differences. Everything from university academics to Iranian dissidents. And sometimes this leads to some healthy contentions.

For instance, as mentioned above my foremost interest in Tor is as an open source community. I push heavily toward openness and doing all our work in public. This is somewhat antithetical though to how security and anonymity communities commonly operate. This tug-of-war is good though, with us meeting somewhere in the middle.

What’s your process for doing development work on the tools you’ve worked on? Is it quite collaborative or do you tend to work on your own then feed back communally?

Unfortunately the later. Tor has subcommunities (such as the core codebase, Guardian, Tails, etc) where multiple developers collaborate on a single codebase, but outside of that our person-to-project ratio is rather poor. I’m the sole developer on Stem and Nyx, which makes best practices such as code reviewing unfeasible.

How do you organise development work in the Tor Project? Do you have a lot of autonomy to make decisions?

Nick once called Tor a ‘do-ocracy’, which is to say that the person doing the work makes the decisions. For sections of tor where a single person does all-the-things (such as where I work) the person leading the charge has full autonomy. However, other sections where we have a larger developer population operate on different models.

As for the internal community as a whole we’ve recently started using more formal voting procedures.

What is the balance between feature development and maintenance? Where do new ideas tend to come from?

This varies between projects and I can only speak to Stem and Nyx. Both of these projects were in the development two full years before their initial release, receiving a high degree of automated test coverage before seeing the light of day. As such maintenance has not been an especially large concern. This comes at the obvious cost though of a long development cycle.

What are the main factors that you consider when making design decisions for your Tor Project work? Could you give me an example of a particularly important/interesting case where you had to make a decision, and how you made this?

Hmmm, there’s quite a few different forms of design decisions (architecture, scalability, etc). For Stem the most relevant is API design so I’ll speak to that. To derive Stem’s API I dogfood the library (use it myself for projects), as well as keep an eye on how others are using it.

I have quite a bit of experience when it comes to API design, but really the best way of discovering where rough edges are is to see a library used in practice, and asking yourself how differences in the library could make their code even better.

When you’re making decisions about the projects you’ve worked on for Tor, at the design stages but also in your day-to-day work, do you find that your own personal values, and the values of the organisation, play a big role in these decisions? Do you think "values" are a useful way of talking about technology, and if so, what values do you think the technologies developed by the Tor Project represent?

Hmmm. Can’t say it’s a foremost thought for me. Mostly I hack on code because I find it fun. This is a volunteer hobby for me, after all.

I’ve noticed throughout my software engineering career that there’s a wide range of primary motivations. For some it’s impact – they want to change the world. For others it’s challenge. Personally I don’t lean toward those. My interests is in our community and doing quality work. The magnitude of impact isn’t a prime motivator for me – I don’t care overly much if my work greatly changes the world or not. Rather, I just care that the things I do are done well.

I suppose that’s why I lean toward support and infrastructure roles.

Do you find your other work (e.g. at Amazon) complements your work at Tor, or are they quite separate?

The two synergize well in that tricks I pick up with Tor tend to benefit Amazon and vice versa. For instance, I first discovered the mocking framework Stem now uses during my dayjob, whereas an Amazon CLIs I develop benefited from my work on Nyx.

That said, I do also keep a degree of separation. Tor Cloud was a project to provide Amazon cloud images to simplify relay setup. I made a point of not touching it with a ten foot pole. That said, honestly it’s never really been an issue. Work knows about Tor, Tor knows about work, and neither seems to care particularly much about the other.

What are the challenges of onboarding new staff, especially when the organisation is going through a phase of expansion?

Just the time to discuss 1:1 about what they’d like on the website, get them set up on irc, etc.

The project is Open Source – why do you think this is particularly important, and what benefits and challenges do you find this poses? How does the Tor Project balance the competing views on this – between openness and more traditional approaches to security development?

Open source is necessary for Tor as a matter of trust. Users depend on Tor to keep them safe, both in their private lives and even more critically in oppressive regimes. If Tor were an uninspectable black box would you trust it? I wouldn’t.

Tor’s whole design is architected around distributed trust. No single relay knows your identity, no directory authority can mess with you, and by keeping the code open source we can’t impair your anonymity
either.

Generally speaking we error toward openness. Exceptions only arise when there’s a need for secrecy. For instance, tor-security@ where sensitive security issues are reported. Another is malicious relay detection so bad actors don’t learn how they’re being caught. However, even those become public eventually (security reports once a fix is available, and the bad relay blacklist is largely public).

What would you say are the main ways that you’ve seen the Tor Project change as an organisation since you’ve been involved?

Thanks to Shari (our executive director) Tor organizationally has greatly matured. Far less angst about job security and funding for folks employed the 501c3 side. As for the community side we’ve grown. One growing pain has been decision making as it turns out consensus doesn’t scale. Ever tried getting unanimous agreement from ninety people on a contentious topic? That… doesn’t work. As such we now have a formalized voting procedure for decision making.

Do you see law enforcement as posing barriers to the work of the Tor Project? Do you think they understand Tor and its goals?

Nope. I don’t see law enforcement as an enemy and hope they don’t see us as one either. Roger and others engage with the law enforcement community and we provide tools like Exonerator to make their lives (and lives of relay operators that don’t want kicked down doors) better.

Just speaking for myself, I was glad to see the Silk Road takedown demonstrate that traditional policing (money trails, informants, etc) still work when it comes to bad actors on Tor. Criminal enterprises have always had IP level privacy through botnets. Our goal is to counteract bulk surveillance and provide individual privacy which I hopefully many (though understandably not all) in the law enforcement community can get behind too.

Are you worried about the potential of governments cracking down on Tor and encryption technologies?

Not my top concern. True, the Crypto wars of the nineties demonstrated that governments can take a laughably ill-conceived stance when it comes to encryption, but thankfully the Internet is global. Even if the US takes a backward stance in this regard EU jurisdictions don’t seem to be showing any sign of following suit.

Does working on Tor mean you need to be more careful in your own day-to-day online security practices?

Not in particular. I don’t involve myself with anything highly sensitive so don’t think I’m a particularly juicy target. Just about all my Tor involvement is public anyway.

The media has in the past tended to focus on negative stories about Tor, and Hidden Services in particular. Does this bother you, and do you think how Tor is perceived is a problem? Do you think there are any ways to deal with “misuse” of the network that are appropriate, or possible?

Good questions. While I can certainly see good uses for Hidden Services (such as dissident blogs) I can’t say it’s a feature I’ve found very compelling. As such my work doesn’t focus on it. Lot of others though see promise in it. You’re right that more than anything else within Tor it draws negative press. Kinda irked when that overshadows the rest of what Tor does but oh well, them’s the breaks.

Tor has many uses – from protecting privacy-conscious citizens, to whistleblowing, to fighting censorship in repressive regimes, to fighting corporate surveillance. Which do you personally think are the most important?

For me personally: privacy and free speech. Authoritarianism requires censorship to survive. It’s unsettling but unsurprising to see denouncement of our free press now that we’re getting our own little dose of extremism here in the US. Freedom of information both press and Tor support are necessary for an informed democracy.

The relay operators I’ve spoken to had very diverse political views, but all had very positive views of the project and the main team. How do you keep the community engaged and happy? What challenges do you face in doing this?

Glad to hear it, we love our wonderful relay community too!

Relay operators are just as much a part of our community as developers, activists, and everyone else. By working in the open hopefully they feel included. Actually, just last month our relay community helped me beta test Nyx. They really did a remarkable job putting it through the ringer. Many thanks to ’em for all their help!

What do you think are the main challenges facing the project in the near future?

Funding diversity and promoting positive uses of Hidden Services are a couple that come to mind.

What would you say, with regards to Tor, is the thing that you’re most excited about in the near future?

Pity you didn’t ask me last month. I’d say the release of Nyx. Now brainstorming my next project so we’ll see.

Final question – are you generally optimistic or pessimistic about the future for internet freedom?

In the short term I’m optimistic. Response to the Snowden revelations showed tremendous public interest in defending digital civil liberties.

But long term I’m worried. Not about government or three letter agencies, but advertising. Market forces and Moore’s law makes bulk surveillance both easier and more profitable every year. Maybe I underestimate the public’s desire for privacy, but when offered convenience in exchange for it I’m uncomfortable thinking where we might end up.

Montreal Developer Meeting

It’s the sign of a busy month if this isn’t my top highlight. As always Jon, Gunner, and Alison orchestrated a great meeting. Between discussions hit the sights and tried poutine for the first time.

As a final note the Tor office moved this month. Sadly this means we’re losing an absolutely fantastic mural done for us by Henry. However, we took some high definition photos. Here’s one I’ve rescaled to be usable as a wallpaper or poster…

Oh how I love Sol Duc. Hundreds of miles of trails, white birch groves, and of course sulfuric hotsprings. Rainforest moss lends the woods an unearthly charm that’s truly just delightful.

Highlight of this month for me was a vacation with my dad, first to Port Townsend then Sol Duc. But this has been an interesting month on other fronts too.

Nyx Performance

This month my chief focus has been tuning Nyx. CPU usage is now 20% lower, and far more responsive under verbose logging due to constant time log deduplication. Overall Nyx finally looks ready for release. I’ll probably move forward with a call for beta testers after the dev meeting.

Membership

This month tor ratified a policy for internal list membership. Much of it just formalizes procedures we’ve used for a while, but it also adds a requirement on continued involvement to stay subscribed. Naturally volunteers move on to other projects over time and this perfectly fine, but eventually their membership will be suspended with re-addition fast tracked if they return.

I’m now facilitating discussions with the Vegas leads and Council on how best to determine this. To give us a starting point I put together some fun stats, but numbers alone don’t come anywhere close to answering the question of “how has this person been involved in making tor better in the last six months?”. Discussions ongoing.

Hi all. This month been down in the engine room productionizing Nyx, with a special focus on memory usage. Dropped ~13%, but still not where I want so investigations are ongoing.

Nyx SQLite Cache

Main benefit came from moving cached consensus information from memory to SQLite. Besides the obvious memory benefits this allows the cache to persist between invocations, halving Nyx’s startup time (from 0.7 to 0.3 seconds).

Hi all. This month pretended to be a web developer and made pretty things. Not an area I dabble in often but it was fun doing something artsy.

New Nyx Website

My eyes, they bleed! Made arm’s old site back in my college days and it shows…

I’m delighted to announce Nyx’s new site is now live! New paint job aside our new site describes Nyx’s features, adds a FAQ, and better changelog.

Tor People Page

So many new people. Between newhires and new volunteers Tor is growing quite a bit right now and a common complaint I’ve heard is that our community makes it maddeningly hard to figure out who’s who. For some of us this is intentional, but others it’s not…

“Who’s that arma guy I was talking with on irc? He seemed nice. Ok, thanks. Now which of these meeting attendees is Roger? Great. And, what does he do?“. Multiply that by ninety of us and it’s no wonder we drive our lovely hair-pulling newcomers to early baldness.

For those that would care to take part our people page now provides irc nicks, descriptions, pgp keys, twitter handles, and pictures. Hope this helps ya get a better idea of who those disembodied voices on irc are!