Start archiving microdescriptors and microdescriptor consensuses

Description

We should start archiving microdescriptors and microdescriptor consensuses, even though clients don't use them yet. We need to extend metrics-db for this, including downloading the latest microdescriptor consensus and the referenced microdescriptors every hour. We might also download all microdescriptors known to all authorities once a day, similar to what we plan to do in #2763.

Child Tickets

Change History (7)

We might also download all microdescriptors known to all authorities once a day, similar to what we plan to do in #2763.

Currently, all authorities that generate microdescriptors keep every microdescriptor they have ever generated. We have a ticket for that bug, and nickm has code to fix it, but he can't merge it to master until 0.2.3.1-alpha is released.

We might also download all microdescriptors known to all authorities once a day, similar to what we plan to do in #2763.

Currently, all authorities that generate microdescriptors keep every microdescriptor they have ever generated. We have a ticket for that bug, and nickm has code to fix it, but he can't merge it to master until 0.2.3.1-alpha is released.

Okay. It looks like downloading all descriptors doesn't do what we expected anyway. But that's something to solve in #2763.

Are there any reasons not to start collecting microdescriptor consensuses and referenced microdescriptors? If not, I might start hacking on this next week.

After looking at the microdesc proposals for #3038, I don't see why we should archive microdescriptors or microdesc consensuses. They don't contain any information about the Tor network that isn't already contained in server descriptors and normal consensuses. The only use I can see is to check that generation of microdescriptors and microdesc consensuses is working properly, but that's hardly enough to archive all of them, is it?