Tor Bug Tracker & Wiki: Ticket Queryhttps://trac.torproject.org/projects/tor/query?status=!closed&cc=~catalyst&order=priority
The Tor Project: anonymity onlineen-USTor Bug Tracker & Wiki/images/tor-logo.pnghttps://trac.torproject.org/projects/tor/query?status=!closed&cc=~catalyst&order=priority
Trac 1.2https://trac.torproject.org/projects/tor/ticket/7349
https://trac.torproject.org/projects/tor/ticket/7349#7349: Obfsbridges should be able to "disable" their ORPortWed, 07 Nov 2012 01:03:14 GMTasn<p>
In the future, we will want obfsbridges to only expose their obfsports and not their ORPort, otherwise an adversary can launch an active-scanning attack against the ORPort.
</p>
<p>
We should spec and implement a torrc option that hides the ORPort of obfsbridges.
</p>
<p>
Maybe it should make the ORPort bind on localhost. But what happens if the transport proxy is not on the same host as the ORPort?
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/7349#changeloghttps://trac.torproject.org/projects/tor/ticket/9498
https://trac.torproject.org/projects/tor/ticket/9498#9498: Allow bridge descriptors to contain no address if they are not being publishedFri, 16 Aug 2013 17:03:20 GMTnwf<p>
To strengthen an "isolating proxy"-style approach to client security, I'd like to allow a Tor bridge node to not reveal its external address(es) in its bridge descriptor. The following patch leaves the address as 0.0.0.0 when it's not going to be published:
</p>
<pre class="wiki">diff --git a/src/or/router.c b/src/or/router.c
index 1063eda..30749b9 100644
--- a/src/or/router.c
+++ b/src/or/router.c
@@ -1772,7 +1772,7 @@ router_rebuild_descriptor(int force)
{
routerinfo_t *ri;
extrainfo_t *ei;
- uint32_t addr;
+ uint32_t addr = 0;
char platform[256];
int hibernating = we_are_hibernating();
const or_options_t *options = get_options();
@@ -1780,11 +1780,16 @@ router_rebuild_descriptor(int force)
if (desc_clean_since &amp;&amp; !force)
return 0;
- if (router_pick_published_address(options, &amp;addr) &lt; 0 ||
- router_get_advertised_or_port(options) == 0) {
+ /* If we're not trying to publish our descriptor, it's OK to use 0.0.0.0
+ * as the address therein.
+ */
+ if ((options-&gt;PublishServerDescriptor_ != NO_DIRINFO) &amp;&amp;
+ (router_pick_published_address(options, &amp;addr) &lt; 0 ||
+ router_get_advertised_or_port(options) == 0)) {
/* Stop trying to rebuild our descriptor every second. We'll
* learn that it's time to try again when ip_address_changed()
* marks it dirty. */
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/9498#changeloghttps://trac.torproject.org/projects/tor/ticket/3652
https://trac.torproject.org/projects/tor/ticket/3652#3652: Export clock skew opinion as getinfo commandTue, 26 Jul 2011 20:17:44 GMTmikeperry<p>
Torbutton could make use of a getinfo command to reduce the fingerprinting issues that a user has wrt their Date() object (see <a class="new ticket" href="https://trac.torproject.org/projects/tor/ticket/2940" title="#2940: enhancement: Adapt browser time based on tor's notion of clock skew... (new)">#2940</a>).
</p>
<p>
However, the threading support in Firefox is not good enough for us to easily make a full tor controller that listens for events. It would be much easier if the current skew (ideally down to milliseconds if possible) was available as a getinfo command, so that Torbutton could just query it and update the browser's time offset.
</p>
<p>
I am making this a 'major' enhancement because clock skew is an important issue to address to reduce fingerprinting.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/3652#changeloghttps://trac.torproject.org/projects/tor/ticket/6311
https://trac.torproject.org/projects/tor/ticket/6311#6311: Migrate TOR_SEARCH_LIBRARY to use pkg-configThu, 05 Jul 2012 20:39:05 GMTnickm<p>
TOR_SEARCH_LIBRARY is a hard-to-maintain fragile garden of venomous snakes, all waiting to bite you at the same time.
</p>
<p>
I'm working on a patch to migrate to pkg-config. pkg-config is everywhere these days, and where it isn't, we can fall back to the old thing for a while.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/6311#changeloghttps://trac.torproject.org/projects/tor/ticket/6877
https://trac.torproject.org/projects/tor/ticket/6877#6877: Finally replace all char[] buffers with uint8_t[] buffersMon, 17 Sep 2012 13:49:49 GMTnickm<p>
We started doing this a while back, and covered a lot of the more worrisome cases, but there's lots more to do. Whenever we pass around a chunk of bytes, it should be as an array of an unsigned type. Otherwise we'll keep getting bugs like <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/6861" title="#6861: defect: Undefined behavior in rend_parse_service_authorization() (closed: fixed)">#6861</a>.
</p>
<p>
This is going to have to go API by API, alas.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/6877#changeloghttps://trac.torproject.org/projects/tor/ticket/7144
https://trac.torproject.org/projects/tor/ticket/7144#7144: Implement Bridge Guards and other anti-enumeration defensesThu, 18 Oct 2012 22:51:06 GMTkarsten<p>
<a class="ext-link" href="https://gitweb.torproject.org/torspec.git/tree/proposals/188-bridge-guards.txt"><span class="icon">​</span>Proposal 188</a> specifies Bridge Guards and other anti-enumeration defenses. We should implement this proposal.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/7144#changeloghttps://trac.torproject.org/projects/tor/ticket/9675
https://trac.torproject.org/projects/tor/ticket/9675#9675: Provide feedback mechanism for clock-skew and other bad problemsThu, 05 Sep 2013 09:45:02 GMTlunar<p>
TBB 3.0 currently has a button to copy Tor logs to the clipboard. It's good enough to enable support by knowledgeable people, but it is also good to enable at least a minimal level of self-support.
</p>
<p>
One misconfiguration that can prevent Tor from working is clock-skew. Vidalia made a bright red message out of it. Having some feedback mechanism in TBB 3.0 for similar critical issues would be good.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/9675#changeloghttps://trac.torproject.org/projects/tor/ticket/10059
https://trac.torproject.org/projects/tor/ticket/10059#10059: capture tor log messages before control connection is openedThu, 31 Oct 2013 14:42:56 GMTmcs<p>
While working on ticket <a class="assigned ticket" href="https://trac.torproject.org/projects/tor/ticket/9675" title="#9675: defect: Provide feedback mechanism for clock-skew and other bad problems (assigned)">#9675</a>, Kathy Brade and I discovered that the tor log warnings regarding clock skew are sometimes generated very early as tor starts up. Unfortunately, there is a known problem in Tor Launcher where tor log messages are not captured until Tor Launcher is able to connect to the control port and issue a SETEVENTS command (the Mozilla process control APIs do not provide a way to capture stdout or stderr).
</p>
<p>
One solution is for tor to provide a way to retrieve old log messages (e.g., a new getinfo command) or otherwise provide a way to capture messages that are generated before a control connection is opened. Another option for this specific scenario would be to modify tor to ensure that clock skew is always reported via the status/bootstrap-phase mechanism.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/10059#changeloghttps://trac.torproject.org/projects/tor/ticket/16564
https://trac.torproject.org/projects/tor/ticket/16564#16564: Reject bridge descriptors posted to non-bridge authoritiesSun, 12 Jul 2015 16:54:55 GMTarma<p>
Right now if my bridge descriptor gets uploaded to the directory authorities, poof I'm now a public relay, even if I didn't mean to be.
</p>
<p>
That's not the end of the world, since I am technically offering to be a relay already, and the only difference is that I didn't opt to publish my descriptor myself.
</p>
<p>
But still it seems like we should make the choice explicit inside the descriptor.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/16564#changeloghttps://trac.torproject.org/projects/tor/ticket/16844
https://trac.torproject.org/projects/tor/ticket/16844#16844: Slow clients can't bootstrap because they expire their consensus fetch but then receive all the bytes from it anyway, making them expire their next fetch, putting them in a terrible loopMon, 17 Aug 2015 18:34:24 GMTarma<p>
If you start your Tor client with no cached directory info, and on a slow (high latency) link, you get:
</p>
<pre class="wiki">$ tail -f tordebug-log|grep connected
Aug 09 16:17:12.299 [info] circuit_handle_first_hop(): Next router is $7EA6EAD6FD83083C538F44038BBFA077587DD755~7EA6EAD6FD83083C538 at 194.109.206.212: Not connected. Connecting.
Aug 09 16:17:12.826 [info] connection_edge_process_relay_cell_not_open(): 'connected' received for circid 2747423797 streamid 16685 after 0 seconds.
Aug 09 16:21:57.298 [info] circuit_handle_first_hop(): Next router is $9695DFC35FFEB861329B9F1AB04C46397020CE31~9695DFC35FFEB861329 at 128.31.0.39: Not connected. Connecting.
Aug 09 16:21:59.099 [info] connection_edge_process_relay_cell_not_open(): 'connected' received for circid 4248917890 streamid 42612 after 1 seconds.
Aug 09 16:22:09.711 [info] circuit_handle_first_hop(): Next router is $332CD489177F202570A7021328A17A91BF823889~332CD489177F202570A at 192.150.94.49: Not connected. Connecting.
Aug 09 16:22:09.711 [info] circuit_handle_first_hop(): Next router is $90E9E44FD74B98F87F7573F917AE4AF651B86B4C~90E9E44FD74B98F87F7 at 5.102.146.106: Not connected. Connecting.
Aug 09 16:22:09.712 [info] circuit_handle_first_hop(): Next router is $547C1CDB516798EC66A01F04A5884DCE1A151919~547C1CDB516798EC66A at 87.72.85.217: Not connected. Connecting.
Aug 09 16:22:12.499 [info] connection_edge_process_relay_cell_not_open(): 'connected' received for circid 2850575558 streamid 43745 after 0 seconds.
Aug 09 16:23:33.901 [info] connection_edge_process_relay_cell(): 'connected' received on circid 2850575558 for streamid 43746, no conn attached anymore. Ignoring.
Aug 09 16:24:11.599 [info] connection_edge_process_relay_cell_not_open(): 'connected' received for circid 4148503990 streamid 17036 after 0 seconds.
Aug 09 16:25:34.306 [info] connection_edge_process_relay_cell(): 'connected' received on circid 4148503990 for streamid 17037, no conn attached anymore. Ignoring.
Aug 09 16:26:29.559 [info] connection_edge_process_relay_cell_not_open(): 'connected' received for circid 2948078868 streamid 42748 after 0 seconds.
[...]
</pre><p>
Oh hey, what's this "no conn attached anymore" issue?
</p>
<pre class="wiki">$ grep 43746 tordebug-log
Aug 09 16:22:12.302 [info] connection_ap_handshake_send_begin(): Sending relay cell 1 on circ 2850575558 to begin stream 43746.
Aug 09 16:22:22.299 [debug] circuit_detach_stream(): Removing stream 43746 from circ 2850575558
Aug 09 16:23:33.901 [debug] connection_edge_process_relay_cell(): Now seen 1433 relay cells here (command 4, stream 43746).
Aug 09 16:23:33.901 [info] connection_edge_process_relay_cell(): 'connected' received on circid 2850575558 for streamid 43746, no conn attached anymore. Ignoring.
Aug 09 16:23:35.799 [debug] connection_edge_process_relay_cell(): Now seen 1434 relay cells here (command 2, stream 43746).
Aug 09 16:23:35.799 [info] connection_edge_process_relay_cell(): data cell dropped, unknown stream (streamid 43746).
[...]
</pre><p>
We're hitting the 10-second stream timeout in connection_ap_expire_beginning(), and detaching the stream from the circuit, and presumably trying it again elsewhere.
</p>
<p>
But that last line above, "data cell dropped", is especially bad news for us -- we get the whole answer, and ignore it all, since we sent an 'end' cell changing our mind but the answer was already coming at us.
</p>
<p>
This situation comes up because of the optimistic data feature -- we get the answer to our request bundled with the 'connected' cell, which is a feature except in the case where we canceled (and then forgot about) the stream.
</p>
<p>
For people trying to bootstrap their Tor on a low-bandwidth high-latency network connection, I bet this landmine will be especially frustrating, since you will be clogging your network connection with directory information that you will discard, which in turn will delay the receipt of the other directory information.
</p>
<p>
You can reproduce the bug on your own, even if you have a great network connection, by starting your Tor with "bandwidthrate 2000 bandwidthburst 5000".
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/16844#changeloghttps://trac.torproject.org/projects/tor/ticket/17278
https://trac.torproject.org/projects/tor/ticket/17278#17278: Fix malleable relay cryptoWed, 07 Oct 2015 15:34:35 GMTnickm<p>
This has been an annoyance in our protocol for entirely too long. Once we have a solid proposal (<a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/5460" title="#5460: defect: Write proposal(s) to implement improved relay/circuit crypto authentication (closed: implemented)">#5460</a>) for this, we should implement it posthaste.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/17278#changeloghttps://trac.torproject.org/projects/tor/ticket/29136
https://trac.torproject.org/projects/tor/ticket/29136#29136: PT_LOG and PT_STATUS event fields unspecifedSat, 19 Jan 2019 22:27:06 GMTatagar<p>
Recently Tor added PT_LOG and PT_STATUS events to the spec...
</p>
<p>
<a class="ext-link" href="https://gitweb.torproject.org/torspec.git/commit/?id=3028cf1"><span class="icon">​</span>https://gitweb.torproject.org/torspec.git/commit/?id=3028cf1</a>
<a class="ext-link" href="https://gitweb.torproject.org/torspec.git/commit/?id=b38257e"><span class="icon">​</span>https://gitweb.torproject.org/torspec.git/commit/?id=b38257e</a>
</p>
<p>
Unfortunately the 'pt-spec.txt section 3.3.5' section they mention does not exist, and in looking around I can't find anything that describes what these event fields are defined as ('PT=' 'TYPE=', 'CONNECT=', etc).
</p>
<p>
I started to write a stem parser for these but can't continue until this is done (I can't parse events without knowing what fields they include).
</p>
<p>
David is aware of this and plans to has kindly offered to add the missing info...
</p>
<pre class="wiki">22:24 &lt;+atagar&gt; dgoulet: Your control-spec addition to descript PT_LOG and PT_STATUS
cite a pt-spec section 3.3.4 which does not exist.
22:24 &lt;+atagar&gt; s/descript/describe
22:29 &lt;+atagar&gt; dgoulet: Huh. I'm not spotting anything that lists the keyword
arguments ('PT=' and 'SEVERITY=') so guess the sections simply
missing from the spec. I need that for stem support so please
give me a nudge when the event spec's done. :)
22:59 &lt;+dgoulet&gt; atagar: oh hmmm I'll fix that sorry
23:17 &lt;+atagar&gt; Thanks! Much appreciated. :)
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/29136#changeloghttps://trac.torproject.org/projects/tor/ticket/2628
https://trac.torproject.org/projects/tor/ticket/2628#2628: Be smarter about launching connections to authorities to learn about clock skewFri, 25 Feb 2011 17:32:59 GMTnickm<p>
While applying altf4's code related to bug1074, some possible enhancements came up. They wouldn't be too hard to do.
</p>
<p>
Right now, we notice clock skew for two reasons: a time from a netinfo cell is different from ours, and a time in an HTTP response header is different from ours. In the netinfo case, if the skew came from an authority, we believe it. If not, and we haven't gotten a netinfo from an authority, we launch an OR connection to an authority.
</p>
<p>
In fact, we should be a bit more sophisticated:
</p>
<ul><li>Any authenticated time from an authority should count as "hearing the time from an authority". This includes not only netinfo cells but also authenticated directory connections.
</li><li>Maybe, skew information from regular HTTP responses should also count as "hearing that we are skewed from a non-authority".
</li><li>Instead of keeping track of *whether* we've heard the correct time from an authority, we should keep track of *when* we heard from the authority. In other words, if we last heard about the correct time from an authority an hour ago and somebody else disagrees with them, the authority is probably right. But if we last heard about the correct time a week ago, we might want to ask again.
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/2628#changeloghttps://trac.torproject.org/projects/tor/ticket/5489
https://trac.torproject.org/projects/tor/ticket/5489#5489: Write up a "how to report bugs and security issues, and what happens then" post or FAQMon, 26 Mar 2012 16:18:27 GMTnickm<p>
We should summarize our current security process on a blog post, FAQ entry, or on the contact page. This hasn't gotten enough attention, since everybody's so busy, but
</p>
<p>
We should at the minimum let people know:
</p>
<ul><li>What issues to do this way and what should just go on the bugtracker. And why.
</li><li>How to report bugs in general.
</li><li>What to expect if you report a security issue.
</li><li>Our current issue evaluation and response process, the history thereof.
</li></ul><p>
This should be someplace pretty easy to find. A longer blog post and a shorter faq or contact entry seems smart to me.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/5489#changeloghttps://trac.torproject.org/projects/tor/ticket/6802
https://trac.torproject.org/projects/tor/ticket/6802#6802: Make our config parsing less kludgyMon, 10 Sep 2012 14:59:49 GMTnickm<p>
If our only configuration types were strings, numbers, booleans, and so forth, our existing configuration parsing mechanism would be nifty. But right now, we do entirely too much with STRING, LINELIST, and worst still LINELIST_V.
</p>
<p>
We've become decent at implementing some patterns to work around this, but it would be neat to get much better.
</p>
<p>
So as first steps, I suggest that we:
</p>
<ul><li>Split the parts of the configuration file parsing that handle the abstract bits of config_format_t into their own file.
</li><li>Turn config_type_t into an OO thing, so that it's easier to add more.
</li><li>Teach the linelist types and the smartlist types to be a bit smarter about what they are lists of, so that every list type doesn't need special handling.
</li><li>Make our linelist_v types have an explicit syntax for sections.
</li><li>Implement an additional function to handle the common case of a line that has some positional arguments and some k{=v} arguments.
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/6802#changeloghttps://trac.torproject.org/projects/tor/ticket/7153
https://trac.torproject.org/projects/tor/ticket/7153#7153: Don't require pluggable transport proxies to be SOCKS proxiesFri, 19 Oct 2012 15:01:50 GMTkarsten<p>
(Re-using text from Zack Weinberg for this description.)
</p>
<p>
There are pluggable transport proxies that don't actually act as a SOCKS proxy. For example, StegoTorus has its own configuration; it ignores everything told it in the SOCKS dialogue and always connects to the bridge that it knows about. If you want multiple StegoTorus bridges accessible to your Tor client, you need multiple <code>"ClientTransportPlugin ... exec"</code> specifications. This is only going to get worse when they move away from having everything set up on StegoTorus' command line, which has been direly needed for some time now.
</p>
<p>
Theoretically all of StegoTorus' configuration <em>could</em> be encapsulated in the SOCKS key-value-pairs-in-the-password hack that's described in 180-pluggable-transport.txt, but they never implemented that and they don't want to. They want to rip out all of the SOCKS code, in fact. The way they want it to work is
</p>
<pre class="wiki">Bridge storus1 direct [keyid=...]
ClientTransportPlugin storus1 direct 127.0.0.1:8888
</pre><p>
In this case, <code>'storus1'</code> is *not* a "method", it's a human-readable identifier for the bridge that Tor will be connected to if it starts talking the OR protocol -- with no initial SOCKS exchange! -- on 127.0.0.1:8888.
</p>
<p>
<code>"direct"</code> should also be valid in CMETHOD/SMETHOD lines for the proxy-management protocol, with the same semantics. Zack says he hasn't really thought through how the server side of this stuff ought to work.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/7153#changeloghttps://trac.torproject.org/projects/tor/ticket/8351
https://trac.torproject.org/projects/tor/ticket/8351#8351: Refactor our controller-command/torrc-option processing logic into a data-driven functionWed, 27 Feb 2013 23:57:46 GMTnickm<p>
Throughout our controller protocol and our configuration code, we process things with more or less the following metaformat:
</p>
<pre class="wiki"> Line = PositionalOptions | KWDOptions | BothOptions
BothOptions = PositionalOptions SP KWDOptions
PositionalOptions = PosOption | PosOption SP PositionalOptions
KWDOptions = KWDOption | KWDOption SP KWDOptions
PosOption = QuotedOption | NonSpace
QuotedOption = QString
NonSpace = PChar*
KWDOption = KWDName "=" PosOption
PChar = a printing, nonspace character.
QString = DOUBLE_QUOTE QContent* DOUBLE_QUOTE
SP = One or more whitespace characters
</pre><p>
We should have one generic function that parses a config line or command into a list of positional and keyword arguments, and another that takes such a list, typechecks it, and converts it into a structure (in the same way that config_assign does). The latter function could even _be_ config_assign.
</p>
<p>
In theory, this should make our parsing more convenient, and remove a bunch of duplicate code.
</p>
<p>
(Did I already open a ticket for this? I'm not seeing it if so.)
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/8351#changeloghttps://trac.torproject.org/projects/tor/ticket/9968
https://trac.torproject.org/projects/tor/ticket/9968#9968: Time out quicker on microdesc fetch failures while we're bootstrappingSat, 12 Oct 2013 14:38:00 GMTarma<p>
<a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/9946" title="#9946: defect: Bootstrapping Tor client hangs up on its chosen guards (closed: fixed)">#9946</a> points out that "if one of our three guards is a bust, it will take 120 seconds for requests to it to time out, and even if the other 2 guards are working great, that typically isn't enough to get to 80% bootstrapped. So there will remain cases where we wait 2 minutes to bootstrap -- and maybe this patch even increases the chance of those since any of the three guards can cause them."
</p>
<p>
We should be more eager to either rotate to a different guard, or try another request in parallel, if a) we're not bootstrapped yet, b) we have directory questions we want answers to, and c) we've recently gotten good answers from our other guards.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/9968#changeloghttps://trac.torproject.org/projects/tor/ticket/11966
https://trac.torproject.org/projects/tor/ticket/11966#11966: "Bootstrapped 20%: Asking for networkstatus consensus" is a lie for bridge usersThu, 15 May 2014 09:16:00 GMTarma<p>
When a Tor client that's configured to use a bridge sees
</p>
<pre class="wiki">[notice] Bootstrapped 20%: Asking for networkstatus consensus
</pre><p>
its next plan is actually to send a DIR_PURPOSE_FETCH_SERVERDESC request for the bridge's descriptor. This is surprising.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/11966#changeloghttps://trac.torproject.org/projects/tor/ticket/12131
https://trac.torproject.org/projects/tor/ticket/12131#12131: Measure connectivity patterns between relaysTue, 27 May 2014 01:55:18 GMTarma<p>
<a class="ext-link" href="https://lists.torproject.org/pipermail/tor-relays/2014-May/004598.html"><span class="icon">​</span>https://lists.torproject.org/pipermail/tor-relays/2014-May/004598.html</a> makes me wonder how many relays are firewalling certain outbound ports (and thus messing with connectivity inside the Tor network). It would be great if somebody would start scanning pairs of relays to see which of them can reach each other and which can't, with the goal of understanding how far from a clique our network topology actually is, and then helping with an awareness campaign to correct it if it's a problem.
</p>
<p>
Tools that might be helpful building blocks here:
</p>
<ul><li>Meejah's exitscanner builds circuits, and makes sure it isn't building too many at once. Uses txtorcon and thus twisted. <a class="ext-link" href="https://github.com/meejah/txtorcon/blob/exit_scanner/apps/exit_scanner/guard-exit-coverage.py"><span class="icon">​</span>https://github.com/meejah/txtorcon/blob/exit_scanner/apps/exit_scanner/guard-exit-coverage.py</a>
</li><li>phw's exitmap does something similar, but with stem rather than txtorcon. <a class="ext-link" href="https://gitweb.torproject.org/user/phw/exitmap.git/tree"><span class="icon">​</span>https://gitweb.torproject.org/user/phw/exitmap.git/tree</a>
</li></ul><p>
Other thoughts:
</p>
<ul><li>You likely want to turn on FastFirstHopPK on the client, so it doesn't waste cpu power on handshakes at the first relay.
</li><li>If you make each relay connect to 6000 other relays in succession, and some of the relays can't handle 6000 open file descriptors at once, then you might mistakenly misinterpret "could not extend to that relay" as a property of the link between the relays when actually it's a property of the first relay. One option is to scan 500 and then move on to another first hop. Another option is to declare this a feature, and try to detect which relays can and which can't handle 6000 open file descriptors at once.
</li><li>n<sup>2</sup> where n is 5000 is actually a heck of a lot of circuits. Should you just build circuits forever in the background, or are there some smarter algorithms for finding interesting patterns without making all 25 million circuits? In particular, there will be a background failure rate anyway, from e.g. relays that happen to be overloaded at that moment. So even 25 million circuits won't be enough.
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/12131#changeloghttps://trac.torproject.org/projects/tor/ticket/12377
https://trac.torproject.org/projects/tor/ticket/12377#12377: get_interface_address6() behaviour iff all interface addresses are internalWed, 11 Jun 2014 19:05:22 GMTrl1987<p>
First, let us assume that all network interfaces for IP host that runs Tor instance are internal as judged by <code>tor_addr_is_internal()</code> function.
</p>
<p>
There is the following code in <code>get_interface_address6()</code> function.
</p>
<pre class="wiki"> /* Try to do this the smart way if possible. */
if ((addrs = get_interface_addresses_raw(severity))) {
int rv = -1;
SMARTLIST_FOREACH_BEGIN(addrs, tor_addr_t *, a) {
if (family != AF_UNSPEC &amp;&amp; family != tor_addr_family(a))
continue;
if (tor_addr_is_loopback(a) ||
tor_addr_is_multicast(a))
continue;
tor_addr_copy(addr, a);
rv = 0;
/* If we found a non-internal address, declare success. Otherwise,
* keep looking. */
if (!tor_addr_is_internal(a, 0))
break;
} SMARTLIST_FOREACH_END(a);
SMARTLIST_FOREACH(addrs, tor_addr_t *, a, tor_free(a));
smartlist_free(addrs);
return rv;
}
</pre><p>
Caller will get the last entry from a interface address smartlist. Is this okay?
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/12377#changeloghttps://trac.torproject.org/projects/tor/ticket/12672
https://trac.torproject.org/projects/tor/ticket/12672#12672: Tor log should be accessible while Tor Browser is starting upMon, 21 Jul 2014 22:02:15 GMTEnvite<p>
Sometimes users write to the Help Desk saying that their Tor Browsers do not manage to start. Since it does not start and stops in the initial loading "Tor Status" screen displaying "Connecting to the Tor Network", users are unable to grab a log to send to the Help Desk.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/12672#changeloghttps://trac.torproject.org/projects/tor/ticket/12930
https://trac.torproject.org/projects/tor/ticket/12930#12930: Someone, somewhere needs to unescape pluggable transport "SMETHOD ARGS" arguments.Sat, 23 Aug 2014 04:01:39 GMTyawning<p>
Per pt-spec.txt:
</p>
<pre class="wiki"> - ARGS:K=V,K=V,K=V
If this option is set, the K=V arguments are added to Tor's
extrainfo document. Equal signs and commas must be escaped
with a backslash.
</pre><p>
All of obfs4's server (extra info) document arguments end with a number of equal signs because they are Base64 strings.
</p>
<p>
goptlib does the right thing here and escapes the args, so the trailing Base64 padding passed to tor as part of SMETHOD ARGS ends with <code>\\=</code>. The fun here is that, tor does not unescape the ARGS line, so <code>\\=</code> is what ends up in the extrainfo document on BridgeDB.
</p>
<p>
The arguments that appear on obfs4 bridge lines should not be escaped, so someone, somewhere between little-t tor, and the place where the arguments appear on whatever BridgeDB frontend the end user sees, needs to unescape the arguments.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/12930#changeloghttps://trac.torproject.org/projects/tor/ticket/16120
https://trac.torproject.org/projects/tor/ticket/16120#16120: Detect if the network goes downTue, 19 May 2015 17:20:37 GMTdgoulet<p>
Implement a way for tor to detect if the network went down. This has multiple use cases (see list of tickets), one for instance is being able to differenciate between "We couldn't connect to a Guard because the Guard is down" vs "We couldn't connect because the network is down". For a more thorough discussion about Guard and network connectivity: <a class="ext-link" href="https://lists.torproject.org/pipermail/tor-dev/2014-August/007346.html"><span class="icon">​</span>https://lists.torproject.org/pipermail/tor-dev/2014-August/007346.html</a>
</p>
<p>
Furthermore, this also makes a difference when we want to either keep a circuit or kill it depending on if the network went down or not.
</p>
<p>
We can get this kind of information from the OS, we need an API generic enough that allows us to build a compat layer for each OS. Let's also make sure it's integrated with the latest control command <code>GETINFO network-liveness</code> and the event <code>NETWORK_LIVENESS</code>.
</p>
<p>
Tickets that could benefit from this: <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/8239" title="#8239: enhancement: Hidden services should try harder to reuse their old intro points (closed: implemented)">#8239</a>, <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/12595" title="#12595: task: Finalize design for improved guard-node behavior (closed: fixed)">#12595</a>
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/16120#changeloghttps://trac.torproject.org/projects/tor/ticket/16458
https://trac.torproject.org/projects/tor/ticket/16458#16458: torspec references UTC, but tor uses unix time (leap second handling)Sun, 28 Jun 2015 17:05:34 GMTteor<p>
When the various torspec documents specify time, they refer to UTC. But the implementations used by at least Linux, *BSD and OS X are based on the Unix time epoch.
</p>
<p>
This makes a difference to how leap seconds are handled: UTC includes leap seconds, but unix time excludes them.
</p>
<p>
We should:
</p>
<ul><li>ensure that none of the security properties of tor depend on leap seconds either being present or absent, either individually or in aggregate:
<ul><li>every minute is not 60 seconds long (and equivalently for hour, day, week)
</li><li>some epoch times can repeat or be missing
</li><li>UTC and Unix time differ by approximately 30 seconds
</li></ul></li><li>check how the current Linux, BSD, Windows and OS X implementations handle leap seconds (in roughly that order of priority)
</li><li>consider and document tor's handling of leap seconds
</li></ul><p>
See:
</p>
<ul><li><a class="ext-link" href="https://en.wikipedia.org/wiki/Leap_second"><span class="icon">​</span>https://en.wikipedia.org/wiki/Leap_second</a>
</li><li><a class="ext-link" href="https://en.wikipedia.org/wiki/Unix_time"><span class="icon">​</span>https://en.wikipedia.org/wiki/Unix_time</a>
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/16458#changeloghttps://trac.torproject.org/projects/tor/ticket/17848
https://trac.torproject.org/projects/tor/ticket/17848#17848: Tor ignores tunneled connections when checking existing directory downloadsMon, 14 Dec 2015 07:55:27 GMTteor<p>
Tor tries to avoid multiple connections to the same directory server to avoid overloading it.
</p>
<p>
But when checking for existing directory downloads from a server in router_pick_trusteddirserver_impl and router_pick_directory_server_impl, tor ignores connections made via a one-hop ORPort tunnel, and connections made via tor.
</p>
<p>
This is a hard problem to solve, because ORPort connections can be used for multiple purposes, and can be indirect, so we can't just check the address and port like the existing code does.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/17848#changeloghttps://trac.torproject.org/projects/tor/ticket/18509
https://trac.torproject.org/projects/tor/ticket/18509#18509: Summarize our crypto migration plans in one placeWed, 09 Mar 2016 18:47:55 GMTarma<p>
Tor has a bunch of crypto use throughout its design: link encryption (nist-style tls), relay identities (1024-bit rsa, soon to be ed25519), circuit handshakes (ntor, but we still support tap), cell crypto (aes), onion service identities (1024-bit rsa, but soon to be this complicated thing), and probably a few more.
</p>
<p>
People keep misunderstanding where we are in the crypto migration plan ("omg tor still uses 1024-bit rsa"). It would be nice to have it all written out in one place that we can reference.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/18509#changeloghttps://trac.torproject.org/projects/tor/ticket/19068
https://trac.torproject.org/projects/tor/ticket/19068#19068: Write and run a clique reachability test.Mon, 16 May 2016 18:35:01 GMTyawning<p>
It would be useful to know what the full inter-relay connectivity graph looks like, and how far it differs from the "every relay can always reach every other relay" ideal.
</p>
<p>
<a class="ext-link" href="https://www.sba-research.org/wp-content/uploads/publications/NavigaTor_preprint.pdf"><span class="icon">​</span>https://www.sba-research.org/wp-content/uploads/publications/NavigaTor_preprint.pdf</a>
</p>
<p>
This should be something doable with stem, and ideally we can run it periodically/automatically, and use it to do things like reject relays that have extremely poor connectivity.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/19068#changeloghttps://trac.torproject.org/projects/tor/ticket/19531
https://trac.torproject.org/projects/tor/ticket/19531#19531: Major cleanup in our baseXX APIsWed, 29 Jun 2016 14:36:02 GMTdgoulet<p>
Our base16/32/64 APIs are quite inconsistent and a timebomb of possible errors and issues. Here is some recommendation with this:
</p>
<p>
1) We should have for _each_ type of encoding a function that returns the encoded size using a source length. We have such function for baese32 and base64 but they are not consistent:
</p>
<ul><li>base32: <code>base32_encoded_size()</code> returns the size including the NUL byte
</li></ul><ul><li>base64: <code>base64_encode_size()</code> has a different name and does NOT add the NUL byte.
</li></ul><p>
They should all return the size with the extra NUL byte because every <code>_encode()</code> function we have requires it in the first place. The other solution is to have a new function <code>baseXX_encoded_string_size()</code> or something that return the NUL byte and the non string function returns the value without it.
</p>
<p>
Else we end up with this kind of code pattern:
</p>
<pre class="wiki">len = base64_encoded_size() + 1
buf = tor_malloc_zero(len)
ret = base64_encode()
tor_assert(ret == len - 1)
</pre><p>
or the +1 added explicitly like so which is _not_ good.
</p>
<pre class="wiki">base32_encode(serviceid, REND_SERVICE_ID_LEN_BASE32+1,
circuit-&gt;rend_data-&gt;rend_pk_digest, REND_SERVICE_ID_LEN);
</pre><p>
or even better:
</p>
<pre class="wiki">base16_encode(hex, 2*CONDITIONAL_CONSENSUS_FPR_LEN+1,
ds-&gt;v3_identity_digest, CONDITIONAL_CONSENSUS_FPR_LEN);
</pre><p>
2) Source length requirement issue. I think though most of them are fixed except base64 API with ticket <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/17868" title="#17868: defect: base64_decode_nopad() destination buffer length problem (closed: fixed)">#17868</a>.
</p>
<p>
I do see this in the <code>base16_decode</code> which could either be a _hard_ requirement assert or assume that if it gets 9 bytes in srclen, well only 8 are decoded. We have callsites that do NOT check the returned value so...
</p>
<pre class="wiki"> if ((srclen % 2) != 0)
return -1;
</pre><p>
3) Every baseXX_encode/decode should return the amount of bytes that has been encoded or decoded. They also should return <code>ssize_t;</code> but that's another ticket I feel like because it's a large refactoring. Here are the missing pieces:
</p>
<ul><li><code>base16_encode()</code> returns void so it should return <code>int</code> as the encoded length.
</li><li><code>base32_encode()</code> returns void.
</li><li><code>base32_decode()</code> returns 0 on success.
</li></ul><p>
4) We should also have macros for the encoded length computation else we ended up with stuff like this (taken from the prop250 branch).
</p>
<pre class="wiki">#define SR_SRV_VALUE_BASE64_LEN \
(((DIGEST256_LEN - 1) / 3) * 4 + 4)
</pre><p>
or flat values
</p>
<pre class="wiki">#define REND_DESC_COOKIE_LEN_BASE64 22
</pre><p>
This is a static calculation so most of the times that macro would be more useful than the function since it could be used for stack allocation which is a BIG use case of ours.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/19531#changeloghttps://trac.torproject.org/projects/tor/ticket/19987
https://trac.torproject.org/projects/tor/ticket/19987#19987: Unit Test Guard, Middle, Exit, Intro, and Rend node choicesFri, 26 Aug 2016 00:40:19 GMTteor<p>
We could unit test the <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/19973" title="#19973: defect: ReachableAddresses applied too broadly (closed: fixed)">#19973</a> fix to the 0.2.8.6 path selection issue by copying the code from test_choose_random_entry_no_guards,
but running it on:
</p>
<ul><li>choose_good_middle_server
</li><li>choose_good_exit_server_general
</li><li>router_choose_random_node (used by rend_consider_services_intro_points)
</li><li>pick_rendezvous_node
</li></ul><p>
The tests should make sure these functions return any node at random.
In 0.2.8.6, these functions chose nodes using the direct connection reachability rules, which was wrong.
</p>
<p>
We could make these tests simpler and more reliable by setting up a node list with a single node that 0.2.8.6 wouldn't choose, but 0.2.8.7 would. For example:
</p>
<ul><li>an IPv4-only node with ClientUseIPv4 0
</li><li>a node on 9001/9030 with FascistFirewall 1
</li><li>a node on 1.1.1.1 with ReachableAddresses 2.0.0.0/8
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/19987#changeloghttps://trac.torproject.org/projects/tor/ticket/20822
https://trac.torproject.org/projects/tor/ticket/20822#20822: Follow-up tasks for prop271 (new guard API) implementationTue, 29 Nov 2016 18:58:55 GMTnickm<p>
This is a parent ticket to capture follow-up tasks that we (might) want to do in order to improve QOI, usability, security, etc, for our new guard algorithm.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/20822#changeloghttps://trac.torproject.org/projects/tor/ticket/21422
https://trac.torproject.org/projects/tor/ticket/21422#21422: Possibly, learn more network data from GUARD_USABLE_NEVER circuits?Thu, 09 Feb 2017 15:10:43 GMTnickm<p>
In <code>cirucit_send_next_onion_skin()</code>, when the circuit is built but we decide not to use it at all because of the guard state, we don't continue on to the rest of the function, where we count statistics about circuit build times and so on. This could be a mistake; we should revisit it.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/21422#changeloghttps://trac.torproject.org/projects/tor/ticket/21445
https://trac.torproject.org/projects/tor/ticket/21445#21445: Launching Tor Browser from the .dmg should obviously fail or install correclty, not neitherMon, 13 Feb 2017 02:13:56 GMTpastly<p>
Launching Tor Browser from the mounted .dmg (instead of dragging it to the application folder link) causes Tor Browser to seemingly install correctly, especially if you're unfamiliar with it.
</p>
<p>
Doing this will open "Tor Browser" to the default Firefox homepage. There's no onion button in the top left, and the browser can't go to any websites because "The proxy server is refusing connections."
</p>
<p>
The ~/Library/Application Support/TorBrowser-Data directory is created with the files found in the attached file (missing the entire Tor subdirectory).
</p>
<p>
Perhaps interestingly, closing "Tor Browser" and reopening it again from the .dmg causes it to actually get installed correctly, but with an additional window asking if I'd like to install Tor Browser's 3 plugins.
</p>
<p>
Either Tor Browser should fail miserably when opened in this way or it should silently correct the user's mistake and install itself as it normally would before opening.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/21445#changeloghttps://trac.torproject.org/projects/tor/ticket/21951
https://trac.torproject.org/projects/tor/ticket/21951#21951: Helping censored users bootstrap to Tor: Tor launcher improvements and automationFri, 14 Apr 2017 21:26:02 GMTlinda<h1 id="Background">Background</h1>
<p>
<a class="ext-link" href="https://petsymposium.org/2017/papers/issue3/paper2-2017-3-source.pdf"><span class="icon">​</span>Research</a> has shown that many censored users are not able to connect to Tor if Tor is censored in their country. The ones that are able to succeed usually do after multiple attempts and minutes of their time.
</p>
<p>
To make this process faster and successful the first time, we need to be able to differentiate people connecting from different countries | people at risk and not at risk | people who are censored and not censored, tighten the window for error notifications and give advice, and generally make the settings easier to configure. One grand vision is to one day not involve users in toggling network settings, and to ask for consent and just give them one button that connects them to Tor safely and reliably.
</p>
<h1 id="Objective">Objective</h1>
<p>
To make it easier for users to connect to Tor, we're going to make some changes to Tor Launcher. We've broken this effort down into three stages:
</p>
<ol><li>design changes: we're going to make interface-only changes that will help the users.
</li><li>naive automation: we're going to automate the connection process, by some sort of behavior. We haven't decided on what that behavior is yet, but there are multiple ways to do this. One way would be to try a bunch of relays/bridges in a specific order, and stop when one is reachable. Another way would be to try all the relays/bridges at the same time, and return one that works to the user.
</li><li>smart automation: this is a "make it work" button that people can relatively safely click, and it will work for people in most environments with minimized risk. We haven't decided on what that behavior is yet either, but one way to do this is to meek-front the connection, work with bridgeDB and/or some way to identify where the user is from, and give them a relay/bridge that works the first time.
</li></ol>Resultshttps://trac.torproject.org/projects/tor/ticket/21951#changeloghttps://trac.torproject.org/projects/tor/ticket/21952
https://trac.torproject.org/projects/tor/ticket/21952#21952: .Onion everywhere?: increasing the use of onion services through automatic redirects and aliasingFri, 14 Apr 2017 21:43:34 GMTlinda<h1 id="Background">Background</h1>
<p>
People can't remember, or type in onion sites very easily. We should try to fix this somehow.
</p>
<blockquote>
<p>
ilf is experimenting with automatically redirecting Tor users to .onion versions of websites that they visit (because they want more people to visit onion sites and they will do so if it is painless to them). But when users were redirected automatically to an onion site, they freaked out about it because they didn't know what happened, didn't know what onion sites were, and the "https" was dropped.
</p>
</blockquote>
<p>
asn and dgoulet also were trying to find a solution to make onion sites more accessible to use. Specifically, onion addresses are quite long and random-ish, making them hard to remember and hard to type. There were many solutions discussed casually to try and resolve this, but none stood out as a clear winner.
</p>
<h1 id="Discussion">Discussion</h1>
<p>
I like the idea of redirecting users to .onion sites automatically when they type in the websites non-onion address. This way, users don't need to remember anything else, need to type in anything long, or really even know what onion sites are.
</p>
<p>
My suggestion is to follow the https design pattern, and create a similar indicator for .onion sites.
</p>
<p>
<a style="padding:0; border:none" href="https://trac.torproject.org/projects/tor/attachment/ticket/21952/onion-address-idea.png"><img width="600px" src="https://trac.torproject.org/projects/tor/raw-attachment/ticket/21952/onion-address-idea.png" /></a>
</p>
<p>
The proposed solution would be this: when a user types in a website (pad.riseup.net), they would automatically be redirected to the onion site. When this happens, there would be an onion icon next to the address bar (replacing the https lock icon if there is one, or just being there an https lock icon would be if redirection from an http website), similar to that of the https lock icon. The address in the address bar can turned a different color or indicated in some way that this is an alias for the onion site.
</p>
<p>
From my observation, people don't mind automatically being redirected to https sites from http sites, but freak out when redirected from an http/https site to an onion site. I don't think that this is because people know what https is and find the idea comforting (although this can help). I speculate that they don't mind because they don't notice, and the reason why users freaked out at the redirect to onion sites is that they saw the website address visibly change in the address bar.
</p>
<p>
Also--
</p>
<p>
If we want to show users the address of the onion site, we can additionally have a feature to reveal the onion site when the user clicks in the address bar.
</p>
<p>
<a style="padding:0; border:none" href="https://trac.torproject.org/projects/tor/attachment/ticket/21952/onion-address-secondary-idea.png"><img width="600px" src="https://trac.torproject.org/projects/tor/raw-attachment/ticket/21952/onion-address-secondary-idea.png" /></a>
</p>
<blockquote>
<p>
I don't know how I feel about this, since it may just be confusing, and just shock the user later. Users don't know that pad.riseup.net resolves to some numerical IP address, and that isn't displayed to users. So there could be an argument made for just indicating that the address is an alisas and not ever showing the .onion address, either. This will confuse way less of the general population.
</p>
</blockquote>
<h1 id="Considerations">Considerations</h1>
<ul><li>how should the redirect behavior work?
</li><li>how can we implement this without tracking?
</li><li>should we allow users to turn off this redirecting behavior?
</li><li>should we hide the .onion address from the users more so than the proposal above?
</li></ul>Resultshttps://trac.torproject.org/projects/tor/ticket/21952#changeloghttps://trac.torproject.org/projects/tor/ticket/21969
https://trac.torproject.org/projects/tor/ticket/21969#21969: We're missing descriptors for some of our primary entry guardsMon, 17 Apr 2017 11:41:46 GMTasn<p>
Seems like there are still a few cases where bootstrapping will stall because of:
</p>
<pre class="wiki">I learned some more directory information, but not enough to build a circuit: We're missing descriptors for some of our primary entry guards
</pre><p>
It seems to happen mainly with bridges. For me it occured with latest master (<code>b081a7ed2</code>) when I was switching between using private bridges and the default TBB bridges. I also heard reports from s7r about this happening.
</p>
<p>
We last changed this behavior in <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/21415" title="#21415: defect: tor_bug_occurred_: Bug: src/or/entrynodes.c:1845: ... (closed: fixed)">#21415</a>, when we made Tor require that we have the descriptor of 2 bridges before we start building circuits.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/21969#changeloghttps://trac.torproject.org/projects/tor/ticket/21974
https://trac.torproject.org/projects/tor/ticket/21974#21974: Race: Tor declares controlport listener open before it has written its controlcookie to diskMon, 17 Apr 2017 19:45:28 GMTarma<p>
When I start my Tor client, I see this in its logs:
</p>
<pre class="wiki">Apr 17 00:49:35.238 [notice] Opening Control listener on 127.0.0.1:9051
</pre><p>
In fact, controllers like txtorcon and nyx might imagine that they can use this log line as an indication that the control port is up and usable:
<a class="ext-link" href="https://github.com/meejah/txtorcon/blob/master/txtorcon/controller.py#L131"><span class="icon">​</span>https://github.com/meejah/txtorcon/blob/master/txtorcon/controller.py#L131</a>
</p>
<p>
But looking through Tor's code, that log line comes from connection_listener_new() which comes from retry_listener_ports() which comes from retry_all_listeners() which comes from options_act_reversible(), which gets called before options_act(), which is where we call init_control_cookie_authentication().
</p>
<p>
So:
</p>
<p>
A) Is there a recommendation we can make to controllers about how they can know when our control port is ready? I guess the answer right now is "when the control port is open and also when the control cookie file is there"? Is that accurate? Should we worry about races writing/reading the cookie file?
</p>
<p>
B) Should we rearrange the order of stuff in options_act*() so things are in place for the control port when we do the log message indicating that it's open?
</p>
<p>
C) We sure have a bunch of stuff we do at startup now, and the dependency graph of what has to happen before what is complicated, and I bet there are bugs in the current order of things. I've opened <a class="new ticket" href="https://trac.torproject.org/projects/tor/ticket/21975" title="#21975: task: Refactor all the startup stuff in config.c, with dependencies in mind (new)">#21975</a> for this bigger refactor idea.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/21974#changeloghttps://trac.torproject.org/projects/tor/ticket/21975
https://trac.torproject.org/projects/tor/ticket/21975#21975: Refactor all the startup stuff in config.c, with dependencies in mindMon, 17 Apr 2017 19:54:46 GMTarma<p>
Motivated by <a class="new ticket" href="https://trac.torproject.org/projects/tor/ticket/21974" title="#21974: defect: Race: Tor declares controlport listener open before it has written its ... (new)">#21974</a>:
"""
We sure have a bunch of stuff we do at startup now, and the dependency graph of what has to happen before what is complicated, and I bet there are bugs in the current order of things. It would be a swell project for somebody to make a list of all the things we do at startup, and figure out their dependencies, and sort them into "waves" or something, and then refactor the stuff in config.c so it matches the new world order.
"""
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/21975#changeloghttps://trac.torproject.org/projects/tor/ticket/22355
https://trac.torproject.org/projects/tor/ticket/22355#22355: Update dir-spec with client fallback directory mirror attempt and timeout behaviourWed, 24 May 2017 00:57:21 GMTteor<p>
Let's add these lines:
</p>
<p>
Clients try several fallback directory mirrors, and use the first one that connects. Each attempt happens after a short delay, regardless of the state of the previous attempt, until at least one attempt has connected.
</p>
<p>
When several fallback directory mirrors have failed, clients start trying directory authorities in a similar fashion.
</p>
<p>
Somewhere near:
</p>
<p>
<a class="ext-link" href="https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3292"><span class="icon">​</span>https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3292</a>
</p>
<p>
I don't think we designed any explicit timeout behaviour, so we are probably using whatever was there before 0.2.8.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22355#changeloghttps://trac.torproject.org/projects/tor/ticket/22359
https://trac.torproject.org/projects/tor/ticket/22359#22359: Community team and network team are constructing glossaries in parallelWed, 24 May 2017 05:49:08 GMTarma<p>
<a href="https://trac.torproject.org/projects/tor/wiki/doc/community/glossary">https://trac.torproject.org/projects/tor/wiki/doc/community/glossary</a>
</p>
<p>
<a class="ext-link" href="https://gitweb.torproject.org/torspec.git/tree/glossary.txt"><span class="icon">​</span>https://gitweb.torproject.org/torspec.git/tree/glossary.txt</a>
</p>
<p>
I wonder if there's some way to team up!
</p>
<p>
(For bonus points, there is a thing on gitweb called a glossary:
<a class="ext-link" href="https://gitweb.torproject.org/tor-glossary.git/tree/"><span class="icon">​</span>https://gitweb.torproject.org/tor-glossary.git/tree/</a>
but it looks less advanced than either of the above two.)
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22359#changeloghttps://trac.torproject.org/projects/tor/ticket/22408
https://trac.torproject.org/projects/tor/ticket/22408#22408: Refactor functions over 300 lines long.Fri, 26 May 2017 18:12:28 GMTnickm<p>
I think it's reasonable to impose a much smaller limit, but let's start by attacking the worst offenders. cc'ing catalyst because we've talked about this before.
</p>
<p>
It's probably a good idea to use a separate ticket or separate branch for each one.
</p>
<pre class="wiki">300 connection_listener_new
306 networkstatus_set_current_consensus
327 rend_service_receive_introduction
330 ed_key_init_from_file
332 circuit_get_open_circ_or_launch
355 tor_spawn_background
360 router_dump_router_to_string
389 networkstatus_verify_bw_weights
389 parse_socks
399 connection_edge_process_relay_cell
404 circuit_expire_building
449 parse_port_config
535 options_act
541 connection_ap_handshake_rewrite_and_attach
548 router_parse_entry_from_string
638 networkstatus_parse_vote_from_string
973 networkstatus_compute_consensus
1269 options_validate
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/22408#changeloghttps://trac.torproject.org/projects/tor/ticket/22455
https://trac.torproject.org/projects/tor/ticket/22455#22455: Enumerate cases where we want to retry circuits, and correctly balance retry robustness with guard discoveryTue, 30 May 2017 19:14:57 GMTarma<p>
In Tor, and especially in the onion service subsystem, we have a bunch of situations where a circuit could fail and we ought to retry:
</p>
<ul><li>the onion service attempting to connect to the rendezvous point
</li><li>the onion service attempting to publish an hsdesc to the hsdir
</li><li>the client attempts to reach the intro point to send its intro1 cell
</li></ul><p>
If we retry too many times, we open ourselves up to new guard discovery attacks (see prop 247). If we retry too few times, we end up with robustness or reachability problems ("Tor doesn't work").
</p>
<p>
It would be nice to just design the single best retry algorithm, and then apply it to all cases. That way we do the hard design and analysis work once, and we don't end up with extra complexity when we combine multiple retry designs. But I think a single best design might not be possible -- compare the service-side hsdir case, where it might be best to wait a while before retrying, to the rend point case, where waiting a while before retrying is not so good. Maybe that argues that we can get away with two best designs, one in the "online, somebody's waiting on me" case, and the other in the "offline, let's get this done reliably but there's no immediate rush" case?
</p>
<p>
Suggested next step: We should write a proposal, with a section enumerating all of the retry situations that tor has; and a section enumerating what we can learn about where the circuit failures are, and how, and how reliable each is (network failure? last hop faillure? guard failure?); and then a section trying to produce the smallest possible number of good designs such that every retry situation is handled well.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22455#changeloghttps://trac.torproject.org/projects/tor/ticket/22469
https://trac.torproject.org/projects/tor/ticket/22469#22469: tor should probably reject "0x00" in port range specificationsThu, 01 Jun 2017 21:58:12 GMTtoralf<p>
something like
</p>
<pre class="wiki">ExitPolicy reject6 [2a00:1450:4001:0815:0000:0000:0000:200e]:0x00
</pre><p>
should spew an error message.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22469#changeloghttps://trac.torproject.org/projects/tor/ticket/22611
https://trac.torproject.org/projects/tor/ticket/22611#22611: Make TB uninstall instructions more detailed in FAQWed, 14 Jun 2017 22:31:56 GMTpastly<p>
macOS requires special treatment.
</p>
<p>
See <a class="ext-link" href="https://github.com/pastly/webwml"><span class="icon">​</span>https://github.com/pastly/webwml</a> branch macos-tb-uninstall
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22611#changeloghttps://trac.torproject.org/projects/tor/ticket/22739
https://trac.torproject.org/projects/tor/ticket/22739#22739: Make routerinfo_t and routerstatus_t addresses immutable; store overrides in node_tTue, 27 Jun 2017 23:36:43 GMTnickm<p>
See this comment in <code>rewrite_node_address_for_bridge</code>:
</p>
<pre class="wiki"> /* XXXX overridden addresses should really live in the node_t, so that the
* routerinfo_t and the microdesc_t can be immutable. But we can only
* do that safely if we know that no function that connects to an OR
* does so through an address from any source other than node_get_addr().
*/
</pre><p>
Here's how we can do that, in several phases.
</p>
<p>
1) Add an "override orport" tor_addrport_t in node_t which, if set, overrides the advertised ports. Make rewrite_node_address_for_bridge() overrwrite that in addition to the stuff it already overwrites.
</p>
<p>
2) Make the various node_get*_addr() look at that field.
</p>
<p>
3) Rename ri-&gt;addr, ri-&gt;*port, md-&gt;addr, and md-&gt;*port, possibly combining them to use the tor_addrport_t structure. This will break everything that uses them. As we fix those compilation errors, make sure that everything using them to decide where to connect uses node_get*_addr() instead -- specifically, one of the functions modified in 2 above.
</p>
<p>
4) Remove the extra logic in rewrite_node_address_for_bridge().
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/22739#changeloghttps://trac.torproject.org/projects/tor/ticket/23061
https://trac.torproject.org/projects/tor/ticket/23061#23061: crypto_rand_double() should produce all possible outputs on platforms with 32-bit intMon, 31 Jul 2017 04:40:58 GMTteor<p>
On 32-bit platforms, crypto_rand_double() only produces 1 in every 2 million possible values between 0 and 1.
</p>
<p>
This happens because:
</p>
<ul><li>crypto_rand_double() divides a random unsigned int by UINT_MAX
</li><li>an unsigned int on 32-bit platforms is 32 bits
</li><li>the mantissa on a double is 53 bits
</li></ul><p>
So crypto_rand_double() doesn't fill the lower 21 bits with random values.
</p>
<p>
This makes the rep_hist_format_hs_stats() noise more predictable on 32-bit platforms.
</p>
<p>
This fix shouldn't affect the unit tests, because they pass on 64-bit.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/23061#changeloghttps://trac.torproject.org/projects/tor/ticket/23523
https://trac.torproject.org/projects/tor/ticket/23523#23523: Handle extreme values better in add_laplace_noise()Fri, 15 Sep 2017 02:49:42 GMTteor<p>
We think it's impossible to get values this large, but we should still check for them and handle them appropriately.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/23523#changeloghttps://trac.torproject.org/projects/tor/ticket/23677
https://trac.torproject.org/projects/tor/ticket/23677#23677: Tor should log what it thinks the time is sometime(s)Wed, 27 Sep 2017 19:32:47 GMTpastly<p>
Many issues come from incorrect time/date/timezone settings. And not all the time does Tor log about how it believes the user's clock is wrong by X hours and Y minutes.
</p>
<p>
For many linux/macos users we can ask them to tell us the output of <code>date -u</code> as a huge troubleshooting help. But not everyone is capable of that. A maybe shouldn't be expected to be.
</p>
<p>
Two ideas:
</p>
<ol><li>Log what Tor thinks the UTC time, local time, and configured timezone are at startup once
</li></ol><ol start="2"><li>Log periodically (say, every HeartbeatPeriod) what Tor thinks [... all the above ...]
</li></ol><p>
Bonus idea:
</p>
<p>
Can we put the timezone in the log line's timestamp? Would that be enough? Is that dangerous for users? Do we make promises about parse-ability that we'd be breaking?
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/23677#changeloghttps://trac.torproject.org/projects/tor/ticket/23863
https://trac.torproject.org/projects/tor/ticket/23863#23863: When our directory guards don't have each others' microdescs, we should try an authority or fallbackSat, 14 Oct 2017 21:27:19 GMTteor<p>
If our directory guards don't have each others' microdescriptors, we should mark some of them dead.
</p>
<p>
But should we mark the one that won't give us the microdescriptor dead?
Or should we mark the one that we can't get a microdescriptor for dead?
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/23863#changeloghttps://trac.torproject.org/projects/tor/ticket/23968
https://trac.torproject.org/projects/tor/ticket/23968#23968: NoScript icon jumps to the right after updateTue, 24 Oct 2017 06:51:32 GMTgk<p>
We had this issue as part of <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/23724" title="#23724: defect: NoScript restartless update breaks Security Slider and its icon disappears (closed: fixed)">#23724</a> but it still seems to be not fixed. After the update to NoScript 5.1.3 (from 5.1.2) the icon jumps to the right and the following error is visible in the browser console
</p>
<pre class="wiki">CustomizableUI: Widget 'noscript-tbb' not found, unable to move CustomizableUI.jsm:1149
</pre><p>
This got reported on our blog (<a class="ext-link" href="https://blog.torproject.org/comment/272025#comment-272025"><span class="icon">​</span>https://blog.torproject.org/comment/272025#comment-272025</a>).
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/23968#changeloghttps://trac.torproject.org/projects/tor/ticket/24020
https://trac.torproject.org/projects/tor/ticket/24020#24020: Can authorities use multihop circuits rather than direct connections to detect running routers?Thu, 26 Oct 2017 18:40:47 GMTnickm<p>
So, I had an item on the roadmap to "Ensure dirauths check for incoming authentication when verifying ORPorts, if easy".
</p>
<p>
Summary: It's not easy, but it's possible given effort.
</p>
<p>
So, it looks like dirauths don't check for incoming authentication at all when verifying ORPorts. All they do is look at the "last_reachable" or "last_reachable6" fields. Those fields are set from dirserv_orconn_tls_done(), which triggers when we complete an outgoing TLS handshake.
</p>
<p>
The reachability tests are launched with dirserv_single_reachability_test(), which only opens a channel -- it doesn't try to create a circuit at all.
</p>
<p>
If we want to do a test for _incoming_ authentication, it's possible, but we'd need to write some more machinery and think of a workaround for an issue (below). We would need to launch testing circuits through the targetted node, and notice whenever somebody authenticates to _us_ using the node's key. If the circuit succeeds but the node has performed no authentication to us, it must be a bridge. Such tests could be launched on a comparatively slow schedule.
</p>
<p>
There's one other problem with the make-an-incoming-circuit approach: I think that the authority will authenticate to the bridge with its outgoing connection, and so the bridge will already have an authority connection to the authority. I think that the bridge will, when asked to connect to the authority, use that connection instead of creating a new one. Two possible fixes: first, the bridge could stop asking for authentication on incoming connections. Second, the authority could stop providing authentication on outgoing testing connections that it launches for this purpose.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/24020#changeloghttps://trac.torproject.org/projects/tor/ticket/24110
https://trac.torproject.org/projects/tor/ticket/24110#24110: document control protocol router status format surprises when using microdescriptorsWed, 01 Nov 2017 06:31:47 GMTteor<p>
In Tor 0.3.0 (or earlier?), we made clients use microdescriptors by default. This changes the format of NS_CONTROL_PORT routerstatus lines, which breaks pytorctl, and therefore the bandwidth authorities.
</p>
<p>
Apparently, this issue can be resolved by setting UseMicrodescriptors 0 or using 0.2.9.
</p>
<p>
edit: Let's update control-spec.txt to document the current (surprising) behavior and track further behavior improvements in other tickets.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/24110#changeloghttps://trac.torproject.org/projects/tor/ticket/24259
https://trac.torproject.org/projects/tor/ticket/24259#24259: Simulate out-of-disk situations in chutney and/or unit tests?Mon, 13 Nov 2017 06:47:01 GMTarma<p>
Tor has a bunch of weird bugs when it runs out of disk space.
</p>
<p>
One of the crazy ones that has been driving us mad for more than a decade is that relays somehow decide to start publishing a new descriptor once per second, forever.
</p>
<p>
Can we make a version of the unit tests where all of the disk interactions act like the disk is full?
</p>
<p>
Can we make a version of the chutney network tests where the disk pretends to be full?
</p>
<p>
This idea seems like a great one to add to our testing toolbelt.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/24259#changeloghttps://trac.torproject.org/projects/tor/ticket/24300
https://trac.torproject.org/projects/tor/ticket/24300#24300: Failed to find node for hop #1 of our path. Discarding this circuitWed, 15 Nov 2017 15:48:21 GMTdgoulet<p>
Been two days in a row with git master (033-alpha), when I boot up my computer and tor starts and fails to bootstrap. It takes around 8 minutes before it finally work.
</p>
<p>
The particularity I have with my tor client is that I pin my <code>EntryNodes</code> with one specific Guard for testing purposes and I stumbled on this issue.
</p>
<p>
Attached to the ticket are the info logs from the start of tor up to the 100% bootstrap.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/24300#changeloghttps://trac.torproject.org/projects/tor/ticket/24367
https://trac.torproject.org/projects/tor/ticket/24367#24367: Changing pluggable transports (during start-up) in Tor Browser is brokenTue, 21 Nov 2017 11:40:18 GMTgk<p>
The fix for <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/23347" title="#23347: defect: Using bridges or switching to bridges sometimes does not work with tor ... (closed: fixed)">#23347</a> made bridges work again in tor but did not fully fix the problem/introduced a new issue: changing pluggable transports during start-up of Tor Browser is broken. Steps to reproduce are:
</p>
<p>
1) Start Tor Browser and select e.g. <code>obfs3</code> as a transport
2) Before finally connecting, click the <code>Cancel</code> button
3) Reconfigure the censorship bypass settings and choose, e.g. <code>obfs4</code>
4) Resume starting. This time you might get an error (indicating that it's still an obfs3 bridge your tor wants to connect to) and/or just a stalled bootstrap process.
</p>
<p>
Even worse, after quitting the browser and restarting the start-up process is still broken: it does not proceed and one only gets
</p>
<pre class="wiki">Nov 21 11:55:59.000 [notice] Ignoring directory request, since no bridge nodes are available yet.
</pre><p>
messages in the terminal.
</p>
<p>
The first bad commit is 93a8ed3b83b5f20768562ca2aff4eba7aca667d8.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/24367#changeloghttps://trac.torproject.org/projects/tor/ticket/24453
https://trac.torproject.org/projects/tor/ticket/24453#24453: Reduce log spam when child process creation failedTue, 28 Nov 2017 07:41:09 GMTcypherpunks<p>
One ERR instead of a lot of WARN would be enough.
</p>
<pre class="wiki">Tor WARN: Failed to create child process TorBrowser\Tor\PluggableTransports\obfs4proxy: The process creation has been blocked.
Tor WARN: Managed proxy at 'TorBrowser\Tor\PluggableTransports\obfs4proxy' failed at launch.
Tor NOTICE: Switching to guard context "bridges" (was using "default")
Tor WARN: Failed to create child process TorBrowser\Tor\PluggableTransports\obfs4proxy: The process creation has been blocked.
Tor WARN: Managed proxy at 'TorBrowser\Tor\PluggableTransports\obfs4proxy' failed at launch.
Tor WARN: Failed to create child process TorBrowser\Tor\PluggableTransports\obfs4proxy: The process creation has been blocked.
Tor WARN: Managed proxy at 'TorBrowser\Tor\PluggableTransports\obfs4proxy' failed at launch.
Tor WARN: Failed to create child process TorBrowser\Tor\PluggableTransports\obfs4proxy: The process creation has been blocked.
Tor WARN: Managed proxy at 'TorBrowser\Tor\PluggableTransports\obfs4proxy' failed at launch.
Tor NOTICE: Delaying directory fetches: No running bridges
Tor WARN: We were supposed to connect to bridge '85.17.30.79:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '[2001:470:b381:bfff:216:3eff:fe23:d6c3]:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.10:15937' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '109.105.109.147:13764' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.12:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.9:12166' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '85.31.186.26:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '109.105.109.165:10527' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.9:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.13:16815' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '38.229.1.78:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '37.218.245.14:38224' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.10:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '192.99.11.54:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.13:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '38.229.33.83:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.10:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.12:4304' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.9:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.11:80' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.11:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '154.35.22.11:16488' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '37.218.240.34:40035' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '85.31.186.98:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '192.95.36.142:443' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
Tor WARN: We were supposed to connect to bridge '83.212.101.3:50002' using pluggable transport 'obfs4', but we can't find a pluggable transport proxy supporting 'obfs4'. This can happen if you haven't provided a ClientTransportPlugin line, or if your pluggable transport proxy stopped running.
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/24453#changeloghttps://trac.torproject.org/projects/tor/ticket/24465
https://trac.torproject.org/projects/tor/ticket/24465#24465: Snowflake broken if no libatomic on hostTue, 28 Nov 2017 23:34:27 GMTisabela<p>
OS: Debian stretch
</p>
<p>
TB alpha 7.5a8
</p>
<p>
Tor Browser logs:
</p>
<p>
<a class="ext-link" href="https://share.riseup.net/#H0k8k6BxXvsOGMq2CODCfg"><span class="icon">​</span>https://share.riseup.net/#H0k8k6BxXvsOGMq2CODCfg</a>
</p>
<p>
steps to reproduce error:
</p>
<ol><li>click on configure
</li></ol><ol start="2"><li>select snowflake
</li></ol><ol start="3"><li>try to connect
</li></ol><ol start="4"><li>error shows up (screenshot: https://trac.torproject.org/projects/tor/attachment/ticket/24465/error-screenshot-snowflake-missing.png)
</li></ol>Resultshttps://trac.torproject.org/projects/tor/ticket/24465#changeloghttps://trac.torproject.org/projects/tor/ticket/25061
https://trac.torproject.org/projects/tor/ticket/25061#25061: Relays consider it a bootstrapping failure if they can't extend for somebody else's circuitMon, 29 Jan 2018 01:34:33 GMTarma<p>
Say you have a relay that is up and listed as non-slow in the consensus. Due to the current overload (<a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/24902" title="#24902: enhancement: Denial of Service mitigation subsystem (closed: fixed)">#24902</a>), this relay is getting many many circuit requests per second. Due to bug <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/24767" title="#24767: enhancement: All relays are constantly connecting to down relays and failing over ... (closed: fixed)">#24767</a>, we will make a huge number of connection attempts to other relays that are down, because as soon as we get a "connection refused", we will get another circuit request that triggers another connection attempt.
</p>
<p>
So when your relay restarts, since it's still in the consensus and clients still think it's usable, it will immediately get flooded with circuit requests, causing these connection attempts to resume.
</p>
<p>
And Tor calls every one of those connection attempts a bootstrapping attempt, even if there are no origin circuits related to that connection attempt.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25061#changeloghttps://trac.torproject.org/projects/tor/ticket/25381
https://trac.torproject.org/projects/tor/ticket/25381#25381: Add crypto_rand_double_sign() in C and RustWed, 28 Feb 2018 04:11:35 GMTteor<p>
We need a function that returns 1.0 or -1.0 with equal probability, so we can avoid weird tricks that waste floating point precision.
</p>
<p>
Since we want to use this in the laplace and guassian distributions, it needs to be implemented in both C and Rust.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25381#changeloghttps://trac.torproject.org/projects/tor/ticket/25504
https://trac.torproject.org/projects/tor/ticket/25504#25504: Find more generic ways to handle smartlist_t/Vec<T> between C and RustWed, 14 Mar 2018 20:04:32 GMTisis<p>
From <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/25368" title="#25368: defect: Update the Tor Rust coding standards for new types (closed: fixed)">#25368</a>, we discussed having a possibly more generic and/or more rusty way to handle our <code>smartlist_t</code>s in C (and whatever underlying types the smartlist contains). Right now we have a <code>Stringlist</code> type in <code>src/rust/smartlist/smartlist.rs</code>, which is a Rust representation of <code>smartlist_t</code> using C types, and then we have a conversion between that and a <code>Vec&lt;String&gt;</code>:
</p>
<div class="wiki-code"><div class="code"><pre><span class="k">pub</span><span class="w"> </span><span class="k">trait</span><span class="w"> </span>Smartlist<span class="o">&lt;</span>T<span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="k">fn</span> <span class="nf">get_list</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Vec</span><span class="o">&lt;</span>T<span class="o">&gt;</span><span class="p">;</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="cp">#[repr(C)]</span><span class="w">
</span><span class="k">pub</span><span class="w"> </span><span class="k">struct</span> <span class="nc">Stringlist</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="k">pub</span><span class="w"> </span>list: <span class="o">*</span><span class="k">const</span><span class="w"> </span><span class="o">*</span><span class="k">const</span><span class="w"> </span>c_char<span class="p">,</span><span class="w">
</span><span class="k">pub</span><span class="w"> </span>num_used: <span class="nc">c_int</span><span class="p">,</span><span class="w">
</span><span class="k">pub</span><span class="w"> </span>capacity: <span class="nc">c_int</span><span class="p">,</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="k">impl</span><span class="w"> </span>Smartlist<span class="o">&lt;</span><span class="nb">String</span><span class="o">&gt;</span><span class="w"> </span><span class="k">for</span><span class="w"> </span>Stringlist<span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="k">fn</span> <span class="nf">get_list</span><span class="p">(</span><span class="o">&amp;</span><span class="bp">self</span><span class="p">)</span><span class="w"> </span>-&gt; <span class="nb">Vec</span><span class="o">&lt;</span><span class="nb">String</span><span class="o">&gt;</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="c1">// [...]
</span><span class="w"> </span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></pre></div></div><p>
I have not thought about this nearly as much as komlo has, but maybe one way to do it is to have direct conversion between a <code>smartlist_t</code> and a <code>Vec&lt;T&gt;</code>, where <code>T</code> is probably an opaque pointer to whatever type in C, <em>or</em> <code>T</code> is only allowed to be a <code>String</code> which we've copied from a non-NULL <code>char*</code> (e.g. <code>impl From&lt;Stringlist&gt; for Vec&lt;String&gt;</code>, or something, and then keep <code>Stringlist</code> private since internally it's a bunch of C types that we don't want propagating into our more Rusty code).
</p>
<p>
Another idea might be to only handle <code>Vec&lt;T&gt;</code>-like things in Rust (if/when we move to the Rust-is-required phase), since we already have a nice datatype there, and then provide safe interfaces for C code to do all the things with/to the vectors that it currently does. (This sounds easier and more maintainable to me.)
</p>
<p>
We should probably brainstorm other ideas of how we're going to do this generically moving forward, because our C code uses smartlists everywhere.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25504#changeloghttps://trac.torproject.org/projects/tor/ticket/25507
https://trac.torproject.org/projects/tor/ticket/25507#25507: Write a guide for groups planning to submit big patches to Tor.Thu, 15 Mar 2018 09:20:07 GMTnickm<p>
This is a master ticket for a document to explain to people or groups who are thinking of landing large branches in the Tor source. It should explain how to do this in an effective and smooth way, go over our practices, and try to prevent as much frustration as possible.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25507#changeloghttps://trac.torproject.org/projects/tor/ticket/25519
https://trac.torproject.org/projects/tor/ticket/25519#25519: move away from asciidocThu, 15 Mar 2018 23:44:21 GMTdkg<p>
asciidoc is the only part of the tor build process that requires python 2. python 2 is deprecated upstream and is approaching end-of-life.
</p>
<p>
Over in <a class="ext-link" href="https://bugs.debian.org/892830"><span class="icon">​</span>https://bugs.debian.org/892830</a> , i asked about upgrading asciidoc to move to python3, but apparently asciidoc itself is on life support, and upstream is encouraging everyone to move to asciidoctor instead (which introduces a ruby dependency) or to use something else like pandoc.
</p>
<p>
in any case, presumably we want the documentation to stay buildable, so i encourage switching away from asciidoc.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25519#changeloghttps://trac.torproject.org/projects/tor/ticket/25713
https://trac.torproject.org/projects/tor/ticket/25713#25713: "DisableNetwork is set" log message in Tor Browser scares/confuses usersWed, 04 Apr 2018 20:16:07 GMTarma<p>
When Tor Browser has trouble connecting, we tell users to go look at the logs. The first thing they see in their logs is something like
</p>
<pre class="wiki"> 13/06/2017 11:31:29.600 [NOTICE] DisableNetwork is set. Tor will not make
or accept non-control network connections. Shutting down all existing
connections.
</pre><p>
and if they're hunting for a log message that gives them a hint about why Tor doesn't work, that one is easy to find and seems really related to why their tor might not work.
</p>
<p>
So we have this recurring event where users come to us and say "My Tor Browser isn't working, it says DisableNetwork is set."
</p>
<p>
Now, Tor Browser legitimately starts Tor with DisableNetwork set, so Tor Browser will have time to ask the user whether they want to use bridges or proxies or etc before their Tor starts interacting with the network. So "well stop using that option then" is hopefully not our best plan.
</p>
<p>
But I wonder if there is a smarter way to have that phrased in the logs, so users have a better chance of guessing correctly what is going on.
</p>
<p>
(Long term we want to not send Tor Browser users to go read Tor's logs. But we're not there yet either.)
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25713#changeloghttps://trac.torproject.org/projects/tor/ticket/25823
https://trac.torproject.org/projects/tor/ticket/25823#25823: Tor Launcher inconsistently sets TZ=UTC for tor processMon, 16 Apr 2018 21:35:06 GMTcatalyst<p>
Some investigations done as part of <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/25511" title="#25511: enhancement: Expose TZ info on control port for better debugging of time errors (closed: implemented)">#25511</a> suggest that the first time Tor Launcher runs tor, it either unsets or fails to change <code>TZ</code>. If tor crashes or needs to be restarted for some reason, then it sets <code>TZ=UTC</code>. My summary of one instance of this behavior is at <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/25511#comment:32" title="#25511: enhancement: Expose TZ info on control port for better debugging of time errors (closed: implemented)">ticket:25511#comment:32</a>.
</p>
<p>
Tor Launcher should probably leave the time zone alone when starting tor, so tor can detect what local time for the machine is. We should also consider the privacy impact of revealing a user's timezone through logging (see <a class="new ticket" href="https://trac.torproject.org/projects/tor/ticket/18112" title="#18112: defect: TorButton logs + Tor logs = timezone leak (new)">#18112</a> for one instance of this concern).
</p>
<p>
In any case Tor Launcher should probably be consistent about which timezone it starts tor in.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/25823#changeloghttps://trac.torproject.org/projects/tor/ticket/26359
https://trac.torproject.org/projects/tor/ticket/26359#26359: DoS and timed attacks via unencrypted network time protocolsTue, 12 Jun 2018 17:44:40 GMTtime_attacker<p>
If a device relies on NTP (or any other unencrypted network time protocol), ISP or other party in the middle can manipulate unencrypted packages to set wrong time. Tor relies on correct time, so ISP can deny Tor usage any time it wants to. Moreover, attacker controlling the ISP (government or hackers compromising ISP's server) can manipulate time on tor-using device, assisting attacks that involve wrong time.
</p>
<p>
Embedded systems like routers have no real-time clock hardware and need to set time via network. PCs are often configured to synchronize time via NTP.
</p>
<p>
Tor should have other way to set the time it needs. It could set time from directory servers and known relays.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/26359#changeloghttps://trac.torproject.org/projects/tor/ticket/27248
https://trac.torproject.org/projects/tor/ticket/27248#27248: Can we make our node-related structures more efficient?Tue, 21 Aug 2018 16:58:08 GMTnickm<p>
Our profiles say that we're using a lot of RAM for our directory stuff. Some of that is for silliness like keeping text of directory stuff in RAM (yuck), but surely some of it is due to all our node_t/microdesc_t/routerstatus_t stuff. Can we make that smaller?
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/27248#changeloghttps://trac.torproject.org/projects/tor/ticket/27351
https://trac.torproject.org/projects/tor/ticket/27351#27351: DisableNetwork is not unset if bootstrapping failsMon, 27 Aug 2018 21:56:09 GMTtraumschule<p>
Reloading TB's Tor instance disables network and keeps it turned off because it fails to attach to the SocksPort.
One has to change the SocksPort, enable the network and change back the SocksPort with nyx:
</p>
<pre class="wiki">&gt;&gt;&gt; signal reload
250 OK
&gt;&gt;&gt; getconf disablenetwork
250 DisableNetwork=1
&gt;&gt;&gt; setconf disablenetwork=0
553 Unable to set option: Failed to bind one of the listener ports.
&gt;&gt;&gt; setconf socksport=9152
250 OK
&gt;&gt;&gt; setconf disablenetwork=0
250 OK
&gt;&gt;&gt; setconf socksport=9150
250 OK
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/27351#changeloghttps://trac.torproject.org/projects/tor/ticket/27604
https://trac.torproject.org/projects/tor/ticket/27604#27604: Relocating the Tor Browser directory is broken with Tor Browser 8Mon, 10 Sep 2018 11:18:59 GMTgk<p>
In Tor Browser 7 users were able to extract Tor Browser in directory <code>foo</code> run it and copy it over to <code>bar</code> and run it. Starting with Tor Browser 8 this does not seem to be possible anymore. See, for instance, the respective report on our blog: <a class="ext-link" href="https://blog.torproject.org/comment/277011#comment-277011"><span class="icon">​</span>https://blog.torproject.org/comment/277011#comment-277011</a>.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/27604#changeloghttps://trac.torproject.org/projects/tor/ticket/27661
https://trac.torproject.org/projects/tor/ticket/27661#27661: use C99 bool from stdbool.h instead of int everywhereWed, 12 Sep 2018 13:42:35 GMTcyberpunksResultshttps://trac.torproject.org/projects/tor/ticket/27661#changeloghttps://trac.torproject.org/projects/tor/ticket/27662
https://trac.torproject.org/projects/tor/ticket/27662#27662: refactor networkstatus_parse_vote_from_string()Wed, 12 Sep 2018 14:31:30 GMTcyberpunks<p>
As of <a class="ext-link" href="https://gitweb.torproject.org/tor.git/tree/src/feature/nodelist/routerparse.c?id=1c62adb65baa99c92f937318c452955306301643#n3395"><span class="icon">​</span>1c62adb65baa99c92f937318c452955306301643</a>, the function is 628 lines long. It could be split into calling 3 helper functions instead.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/27662#changeloghttps://trac.torproject.org/projects/tor/ticket/28014
https://trac.torproject.org/projects/tor/ticket/28014#28014: Windows support for Travis CIFri, 12 Oct 2018 00:49:54 GMTahf<p>
Travis CI just announced Windows support in:
</p>
<p>
<a class="ext-link" href="https://blog.travis-ci.com/2018-10-11-windows-early-release"><span class="icon">​</span>https://blog.travis-ci.com/2018-10-11-windows-early-release</a>
</p>
<p>
This might be worth looking into since I think our AppVeyor setup have been a bit flaky.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/28014#changeloghttps://trac.torproject.org/projects/tor/ticket/28027
https://trac.torproject.org/projects/tor/ticket/28027#28027: Tor keeps opening circuits while waiting for bridge descriptorsFri, 12 Oct 2018 21:31:06 GMTdgoulet<p>
From someone in #tor-dev (<code>Tovok7</code>), they migrated an HS from 029 to 034 and some feature broke.
</p>
<p>
Tor, configured with an HS, starts fine, bootstraps and all is good.
</p>
<p>
Then, through the control port, <code>setconf UseBridge=1 Bridge=...</code> created these logs:
</p>
<pre class="wiki">2018-10-12 16:18:54.440 I/TorPlugin: Enabling network, using bridges
2018-10-12 16:18:54.456 I/TorPlugin: NOTICE Switching to guard context "bridges" (was using "default")
2018-10-12 16:18:54.608 I/TorPlugin: NOTICE Delaying directory fetches: No running bridges
2018-10-12 16:18:55.031 I/TorPlugin: WARN Error launching circuit to node [scrubbed] for service [scrubbed].
2018-10-12 16:18:55.032 I/TorPlugin: WARN Error launching circuit to node [scrubbed] for service [scrubbed].
2018-10-12 16:18:55.616 I/TorPlugin: OR connection LAUNCHED $02069A3C5362476936B62BA6F5ACC41ABD573A9B
2018-10-12 16:18:55.616 I/TorPlugin: OR connection LAUNCHED $5A2D2F4158D0453E00C7C176978D3F41D69C45DB
2018-10-12 16:18:55.616 I/TorPlugin: OR connection LAUNCHED $B31A7DAD9AACEDDB9915A16617BB8F06BA429D6B
2018-10-12 16:18:55.642 I/TorPlugin: WARN Hidden service [scrubbed] exceeded launch limit with 13 intro points in the last 13 seconds. Intro circuit launches are limited to 10 per 300 seconds.
2018-10-12 16:18:55.642 I/TorPlugin: WARN Service configured in [EPHEMERAL]:
2018-10-12 16:18:55.642 I/TorPlugin: WARN Intro point 0 at [scrubbed]: no circuit
2018-10-12 16:18:55.643 I/TorPlugin: WARN Intro point 1 at [scrubbed]: no circuit
2018-10-12 16:18:55.643 I/TorPlugin: WARN Intro point 2 at [scrubbed]: no circuit
2018-10-12 16:18:55.643 I/TorPlugin: WARN Intro point 3 at [scrubbed]: no circuit
2018-10-12 16:18:55.643 I/TorPlugin: WARN Intro point 4 at [scrubbed]: no circuit
2018-10-12 16:18:56.652 I/TorPlugin: OR connection CONNECTED $02069A3C5362476936B62BA6F5ACC41ABD573A9B
2018-10-12 16:18:57.013 I/TorPlugin: NOTICE new bridge descriptor 'pointingRespighi' (fresh): [scrubbed]
2018-10-12 16:18:57.014 I/TorPlugin: NOTICE Our directory information is no longer up-to-date enough to build circuits: We're missing descriptors for 1/2 of our primary entry guards (total microdescriptors: 6457/6457).
2018-10-12 16:18:57.080 I/TorPlugin: OR connection CONNECTED $5A2D2F4158D0453E00C7C176978D3F41D69C45DB
2018-10-12 16:18:57.184 I/TorPlugin: OR connection CONNECTED $B31A7DAD9AACEDDB9915A16617BB8F06BA429D6B
2018-10-12 16:18:57.586 I/TorPlugin: NOTICE new bridge descriptor 'AlliumGermanicus' (fresh): [scrubbed]
2018-10-12 16:18:57.641 I/TorPlugin: NOTICE Bridge 'nonLinearGeometry' has both an IPv4 and an IPv6 address. Will prefer using its IPv4 address ([scrubbed]) based on the configured Bridge address.
2018-10-12 16:18:57.641 I/TorPlugin: NOTICE new bridge descriptor 'nonLinearGeometry' (fresh): [scrubbed]
2018-10-12 16:18:57.641 I/TorPlugin: NOTICE We now have enough directory information to build circuits.
2018-10-12 16:19:00.201 I/TorPlugin: NOTICE Tor has successfully opened a circuit. Looks like client functionality is working.
</pre><p>
It appears that the HS tried to open intro points even though tor didn't have bridge descriptors (guard state got switched).
</p>
<p>
The HS subsystem is safeguarded by this check (for circuit events):
</p>
<pre class="wiki"> if (router_have_consensus_path() == CONSENSUS_PATH_UNKNOWN ||
!have_completed_a_circuit()) {
return;
}
</pre><p>
In other words, if we can't open circuits, tor will never proceed with HS service circuits.
</p>
<p>
The main theory, discussed with armadev, can be deduced with the three first log line:
</p>
<pre class="wiki">2018-10-12 16:18:54.456 I/TorPlugin: NOTICE Switching to guard context "bridges" (was using "default")
2018-10-12 16:18:54.608 I/TorPlugin: NOTICE Delaying directory fetches: No running bridges
2018-10-12 16:18:55.031 I/TorPlugin: WARN Error launching circuit to node [scrubbed] for service [scrubbed].
</pre><p>
First line: Guard context switched to bridges. All is good.
</p>
<p>
Second line: <code>router_have_minimum_dir_info()</code> is called from, actually wherever... It is used quite often in many places including our mainloop. The point is that within that function, we do look at <code>should_delay_dir_fetches()</code> which is the one creating that notice. However, because 1 was returned, we never went to <code>update_router_have_minimum_dir_info()</code> which would have mark that we can't complete circuits (with <code>note_that_we_maybe_cant_complete_circuits()</code>).
</p>
<p>
Third line: Circuit launch failure.
</p>
<p>
Once the guard context was switched, all circuits were marked as unusable (normal) so the HS service has to rebuild all its intro points but the <code>have_completed_a_circuit()</code> was still returning true.
</p>
<p>
Whatever is the cause, there is a clear issue that when we switch guard context, we should _always_ stop circuit creation until the guard state is usable.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/28027#changeloghttps://trac.torproject.org/projects/tor/ticket/28106
https://trac.torproject.org/projects/tor/ticket/28106#28106: Change integration tests from bash to shellThu, 18 Oct 2018 13:26:45 GMTjuga<p>
gman999 reported that tests need to change from bash to shell to include them in openbsd
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/28106#changeloghttps://trac.torproject.org/projects/tor/ticket/29201
https://trac.torproject.org/projects/tor/ticket/29201#29201: Tor bootstrap hangs when offlineTue, 29 Jan 2019 23:09:35 GMTatagar<p>
Hi Nick. When launching a tor process stem uses bootstrap messages to determine when the instance we launch is available. Recently-ish tor changed such that when offline tor bootstrapping does not progress past 0%, printing hundreds of...
</p>
<pre class="wiki">Jan 29 11:36:27.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 11; recommendation warn; host 322C6E3A973BC10FC36DE3037AD27BC89F14723B at 212.83.154.33:8443)
Jan 29 11:36:28.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 12; recommendation warn; host F741E5124CB12700DA946B78C9B2DD175D6CD2A1 at 163.172.154.162:9001)
Jan 29 11:36:28.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 14; recommendation warn; host D71B1CA1C9DC7E8CA64158E106AD770A21160FEE at 185.34.33.2:31415)
Jan 29 11:36:29.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 13; recommendation warn; host F2DFE5FA1E4CF54F8E761A6D304B9B4EC69BDAE8 at 129.13.131.140:443)
Jan 29 11:36:30.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 14; recommendation warn; host 47C42E2094EE482E7C9B586B10BABFB67557030B at 185.220.101.34:20034)
Jan 29 11:36:30.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 14; recommendation warn; host B06F093A3D4DFAD3E923F4F28A74901BD4F74EB1 at 178.17.174.14:9001)
Jan 29 11:36:31.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 15; recommendation warn; host CF6D0AAFB385BE71B8E111FC5CFF4B47923733BC at 154.35.175.225:443)
</pre><p>
There's a few issues with this...
</p>
<ol><li>Poor experience from a user perspective. Deluging the user with hundreds of warnings is pretty unhelpful.
</li></ol><ol start="2"><li>Stem's ability to launch tor processes no longer works when offline.
</li></ol><ol start="3"><li>Stem's integ tests no longer pass when offline. I can sidestep this but first I'd like to confirm if this is the desired behavior from tor or not.
</li></ol>Resultshttps://trac.torproject.org/projects/tor/ticket/29201#changeloghttps://trac.torproject.org/projects/tor/ticket/29432
https://trac.torproject.org/projects/tor/ticket/29432#29432: QuotedString and CString in control-spec.txt technically require escaping ascii 32 (space)Fri, 08 Feb 2019 02:26:34 GMTdcf<p>
control-spec.txt 2.1 <a class="ext-link" href="https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=795420240305a6d67c0f4322993a65da4c7b6f2f#n110"><span class="icon">​</span>says</a>:
</p>
<blockquote class="citation">
<h3 id="a2.1.Descriptionformat">2.1. Description format</h3>
<p>
We use the following nonterminals from RFC 2822: <code>atom</code>, <code>qcontent</code>
...
<code>QuotedString = DQUOTE *qcontent DQUOTE</code>
...
</p>
<h3 id="a2.1.1.Notesonanescapingbug">2.1.1. Notes on an escaping bug</h3>
<p>
<code>CString = DQUOTE *qcontent DQUOTE</code>
</p>
</blockquote>
<p>
RFC 2822 <a class="ext-link" href="https://tools.ietf.org/html/rfc2822#section-3.2.5"><span class="icon">​</span>defines</a> <code>qcontent</code> thus:
</p>
<pre class="wiki">qtext = NO-WS-CTL / ; Non white space controls
%d33 / ; The rest of the US-ASCII
%d35-91 / ; characters not including "\"
%d93-126 ; or the quote character
qcontent = qtext / quoted-pair
</pre><p>
where <code>NO-WS-CTL</code> <a class="ext-link" href="https://tools.ietf.org/html/rfc2822#section-3.2.1"><span class="icon">​</span>expands to</a>
</p>
<pre class="wiki">NO-WS-CTL = %d1-8 / ; US-ASCII control characters
%d11 / ; that do not include the
%d12 / ; carriage return, line feed,
%d14-31 / ; and white space characters
%d127
</pre><p>
In short, <code>qcontent</code> does not include the space character (ascii 32), and so according to a strict reading of the spec, anything that produces a QuotedString or CString has to escape spaces as <code>\ </code> or <code>\040</code>.
</p>
<p>
The reason why RFC 2822 does not require space to be escaped is that the definition of <code>quoted-string</code> is not <code>DQUOTE *qcontent DQUOTE</code> as in control-spec.txt, but also allows whitespace as part of the <code>[FWS]</code> production:
</p>
<pre class="wiki">quoted-string = [CFWS]
DQUOTE *([FWS] qcontent) [FWS] DQUOTE
[CFWS]
</pre><p>
I notice that tor doesn't escape the space character (in <code>esc_for_log</code> and <code>unescape_string</code> for example). IMO tor is doing the right, expected thing and the spec should be clarified.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/29432#changeloghttps://trac.torproject.org/projects/tor/ticket/29777
https://trac.torproject.org/projects/tor/ticket/29777#29777: Rate-limit "Problem bootstrapping" warnings to one every 5 secondsWed, 13 Mar 2019 23:13:40 GMTteor<p>
Let's put a rate-limit on warnings like this:
</p>
<pre class="wiki">Jan 29 11:36:27.000 [warn] Problem bootstrapping. Stuck at 0% (starting): Starting. (Network is unreachable; NOROUTE; count 11; recommendation warn; host 322C6E3A973BC10FC36DE3037AD27BC89F14723B at 212.83.154.33:8443)
</pre>Resultshttps://trac.torproject.org/projects/tor/ticket/29777#changeloghttps://trac.torproject.org/projects/tor/ticket/29779
https://trac.torproject.org/projects/tor/ticket/29779#29779: Stem's integ tests no longer pass when offlineWed, 13 Mar 2019 23:21:15 GMTteor<p>
Once we fix <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/29778" title="#29778: defect: Stem's ability to launch tor processes no longer works when offline (closed: fixed)">#29778</a>, we might need to also fix Stem or Tor so that Stem's integ tests can pass when the network is offline.
</p>
<p>
I don't know if this will require changes in Stem or Tor, or what those changes need to be.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/29779#changeloghttps://trac.torproject.org/projects/tor/ticket/29925
https://trac.torproject.org/projects/tor/ticket/29925#29925: "See My Path" onboarding not shown after tab switchWed, 27 Mar 2019 15:45:06 GMTmcs<p>
This is a spinoff from <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/27484#comment:13" title="#27484: defect: Onboarding: unintuitive not-navigation buttons, starting with &#34;Circuit ... (closed: fixed)">ticket:27484#comment:13</a>.
</p>
<p>
The fancy circuit display onboarding that is invoked in response to the "See My Path" button is not shown at all if the user switches away from the tab while it is loading. We should fix that, but we could also think about how to make the entire experience less confusing given the fact that (1) we wait for the DuckDuckGo page to finish loading before we show the user anything related to onboarding, and (2) that page sometimes takes a long time to load.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/29925#changeloghttps://trac.torproject.org/projects/tor/ticket/30235
https://trac.torproject.org/projects/tor/ticket/30235#30235: Tor hangs when asked to change DisableAllSwap over the control portThu, 18 Apr 2019 03:07:43 GMTteor<pre class="wiki"> self.assertRaisesWith(stem.InvalidArguments, "DisableAllSwap, User cannot be changed while tor's running", controller.set_options, {'User': 'atagar', 'DisableAllSwap': '1'})
</pre><p>
or
</p>
<pre class="wiki"> File "/home/travis/build/tlyu/tor/stem/test/integ/control/controller.py", line 793, in test_set_conf_when_immutable
self.assertRaisesWith(stem.InvalidArguments, "DisableAllSwap cannot be changed while tor's running", controller.set_conf, 'DisableAllSwap', '1')
</pre><p>
We don't know why, because we don't have the tor logs or backtrace (<a class="merge_ready ticket" href="https://trac.torproject.org/projects/tor/ticket/30234" title="#30234: enhancement: Get a stacktrace from tor processes launched by stem (merge_ready)">#30234</a>).
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/30235#changeloghttps://trac.torproject.org/projects/tor/ticket/30580
https://trac.torproject.org/projects/tor/ticket/30580#30580: Tor rejects all POSTDESCRIPTOR controller requestsThu, 23 May 2019 03:58:41 GMTteor<p>
In <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/30091" title="#30091: enhancement: Unify parsing code for control.c (closed: fixed)">#30091</a>, we replaced this code:
</p>
<pre class="wiki"> if (!strcasecmpstart(option, "purpose=")) {
option += strlen("purpose=");
purpose = router_purpose_from_string(option);
if (purpose == ROUTER_PURPOSE_UNKNOWN) {
connection_printf_to_buf(conn, "552 Unknown purpose \"%s\"\r\n",
option);
goto done;
}
}
</pre><p>
With this code:
</p>
<pre class="wiki"> line = config_line_find_case(args-&gt;kwargs, "purpose");
if (line) {
purpose = router_purpose_from_string(line-&gt;value);
connection_printf_to_buf(conn, "552 Unknown purpose \"%s\"\r\n",
line-&gt;value);
goto done;
}
</pre><p>
There's no purpose check any more (<code>if (purpose == ROUTER_PURPOSE_UNKNOWN) {</code>), so Tor rejects all POSTDESCRIPTOR requests.
</p>
<p>
I'm assigning this bug to nickm and cc'ing catalyst, because they were the author and reviewer.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/30580#changeloghttps://trac.torproject.org/projects/tor/ticket/7479
https://trac.torproject.org/projects/tor/ticket/7479#7479: Replace more linked lists with queue.h implementationsThu, 15 Nov 2012 13:31:42 GMTnickm<p>
We've got linked lists scattered through our source. Now that we have a queue.h implementation, it would be neat to use that instead.
</p>
<p>
This is something we can (and should!) do piecemeal; let's not try to do it all with one big patch.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/7479#changeloghttps://trac.torproject.org/projects/tor/ticket/7869
https://trac.torproject.org/projects/tor/ticket/7869#7869: ntor-onion-key is padded with an equal signSat, 05 Jan 2013 20:46:32 GMTrransom<p>
Replying to <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/7867" title="#7867: defect: dir-spec doesn't specify ntor-onion-key (closed: duplicate)">sonu</a>:
</p>
<blockquote class="citation">
</blockquote>
<pre class="wiki">ntor-onion-key Od2Sj3UXFyDjwESLXk6fhatqW9z/oBL/vAKJ+tbDqUU=
</pre><p>
The unnecessary “<code>=</code>” at the end of that string needs to go away <strong>now</strong>, or every Tor client will have to download a thousand or so of them every week forever.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/7869#changeloghttps://trac.torproject.org/projects/tor/ticket/13260
https://trac.torproject.org/projects/tor/ticket/13260#13260: Transform code to cleaner c99 styleFri, 26 Sep 2014 15:16:34 GMTnickm<p>
For <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/13233" title="#13233: defect: Drop c90 compiler support (closed: implemented)">#13233</a>, we added a loose c99 requirement for building Tor. If we decide to keep it through the 0.2.6.x series, we can beautify our code a little.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/13260#changeloghttps://trac.torproject.org/projects/tor/ticket/26105
https://trac.torproject.org/projects/tor/ticket/26105#26105: Perhaps, make test coverage deterministic _within_ linesTue, 15 May 2018 15:58:46 GMTnickm<p>
See <a class="needs_information ticket" href="https://trac.torproject.org/projects/tor/ticket/25907" title="#25907: defect: Highlighting text causes Tor to completely freeze. (needs_information)">#25907</a> and <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/26101" title="#26101: defect: Update cov-diff script to understand * character (closed: fixed)">#26101</a> for background: apparently gcov now has a feature where it can tell us that a line was reached, but not every basic block within the line was reached.
</p>
<p>
Right now, our unit tests have two cases where sometimes a line is completely covered, and sometime the line is only partially covered:
</p>
<pre class="wiki">--- a/workqueue.c.gcov.tmp
+++ /workqueue.c.gcov.tmp
@@ -198,7 +198,7 @@
1: tor_mutex_acquire(&amp;ent-&gt;on_pool-&gt;lock);
1: workqueue_priority_t prio = ent-&gt;priority;
1: if (ent-&gt;pending) {
- 1*: TOR_TAILQ_REMOVE(&amp;ent-&gt;on_pool-&gt;work[prio], ent, next_work);
+ 1: TOR_TAILQ_REMOVE(&amp;ent-&gt;on_pool-&gt;work[prio], ent, next_work);
1: cancelled = 1;
1: result = ent-&gt;arg;
-: }
</pre><pre class="wiki">--- a/compat_pthreads.c.gcov.tmp
+++ /compat_pthreads.c.gcov.tmp
@@ -271,7 +271,7 @@
-: }
1: tvnow.tv_sec = ts.tv_sec;
1: tvnow.tv_usec = (int)(ts.tv_nsec / 1000);
- 1*: timeradd(tv, &amp;tvnow, &amp;tvsum);
+ 1: timeradd(tv, &amp;tvnow, &amp;tvsum);
-:#else /* !(defined(HAVE_CLOCK_GETTIME) &amp;&amp; defined(USE_COND_CLOCK)) */
-: if (gettimeofday(&amp;tvnow, NULL) &lt; 0)
-: return -1;
</pre><p>
Unless coveralls cares about these, I think solving the determinism here is not super-important, though it might be fun from a completionist perspective.
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/26105#changeloghttps://trac.torproject.org/projects/tor/ticket/26076
https://trac.torproject.org/projects/tor/ticket/26076#26076: test_keygen.sh and test_key_expiration.sh fail on WindowsFri, 11 May 2018 02:07:21 GMTsaper<pre class="wiki">FAIL: src/test/test_keygen.sh
=============================
May 11 01:50:30.175 [warn] Path for GeoIPFile (&lt;default&gt;) is relative and will resolve to C:\projects\appveyor\i686-w64-mingw32\&lt;default&gt;. Is this what you wanted?
May 11 01:50:30.175 [warn] Path for GeoIPv6File (&lt;default&gt;) is relative and will resolve to C:\projects\appveyor\i686-w64-mingw32\&lt;default&gt;. Is this what you wanted?
Tor didn't declare that there would be no encryption
FAIL src/test/test_keygen.sh (exit status: 5)
SKIP: src/test/fuzz_static_testcases.sh
</pre><p>
Edit: when we fix this bug, we should revert <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/26830" title="#26830: defect: SKIP test_keygen.sh on Windows (closed: fixed)">#26830</a> and <a class="closed ticket" href="https://trac.torproject.org/projects/tor/ticket/26853" title="#26853: defect: SKIP test_key_expiration.sh on Windows (closed: fixed)">#26853</a>, which skips these tests on Windows
</p>
Resultshttps://trac.torproject.org/projects/tor/ticket/26076#changelog