tor-webwml.git

1) ## translation metadata

2) # Revision: $Revision$

3) # Translation-Priority: 4-optional

4) 
5) #include "head.wmi" TITLE="NLnet Project: Tor for Low Bandwidth Clients"
6) 
7) <div class="main-column">
8) 
9) <!-- PUT CONTENT AFTER THIS TAG -->
10) 
11) <h2>NLnet Project: Tor for Low Bandwidth Clients</h2>
12) <hr />
13) 
14) <p>
15) The Tor anonymity system is currently only usable by internet users who
16) have high-bandwidth connections. Upon the start of the Tor client, a large file
17) with all Tor server descriptions is being downloaded. This file is the so-called Tor
18) Directory and enables the client to choose from the available mix-servers in the Tor network. The
19) download of the full Tor Directory is required by the current Tor protocol. This directory
20) file is too large for users on modem lines or on mobile data networks like GPRS as the initial
21) download that is triggered every time an user logs in takes 10 to 30 minutes over a slow
22) line. As a result, Tor is not usable by modem and mobile users. One of the major goals of
23) the Tor project is to provide secure anonymous internet access to users in dictatorships
24) and repressive states. These locations often have very slow internet connections,
25) either by modem or due to low-bandwidth links to the outside world. By enabling these users to
26) use the Tor network, significant progress can be made towards free communication and
27) information in these countries.
28) </p>
29) 
30) <p>
31) To make Tor usable also for users on low-bandwidth connections, an
32) evolution of the Tor protocol is needed to reduce the initial download size. This new Tor
33) protocol version should change the way a client receives the information for its Tor
34) circuit setup in a way, that the initial download can be performed over a 14.4 kbps modem line
35) in about three minutes. The work to be conducted under the proposal has the ultimate
36) goal of getting the protocol change production ready and propagated to the Tor users
37) within a timeframe of less then 12 months. The resulting software will be published under
38) the 3-Clause BSD license, as all of the Tor code. All deliverables will be fully public.
39) </p>
40) 
41) <p>
42) This project is generously funded by:
43) </p>
44) 
45) <p>
46) <a href="http://www.nlnet.nl/news/2008/20080514-awards.html">
47) <img src="$(IMGROOT)/nlnet-160x60.png" alt="The NLnet foundation" /></a>
48) </p>
49) 
50) <table width="100%" border="0" cellspacing="0" cellpadding="3">

51) <thead>
52) <tr>
53) <th><big>Project</big></th>
54) <th><big>Due Date</big></th>

55) </tr>

56) </thead>

57) 
58) <tr bgcolor="#e5e5e5">
59)   <td>
60)     <b>Deliverable A:</b> Design and evaluation of the protocol change<br />
61)     <small><em>This deliverable covers the detailed design and
62)     simulation-based evaluation of the necessary changes and design
63)     modifications to the current Tor protocol.
64)     The changes in protocol will be relatively substantial, so it requires a careful
65)     evaluation of possible repercussions for the security and anonymity of the Tor network. A
66)     two-month period is planned for this design and evaluation phase, which concludes with an
67)     extensive peer review. Part of Deliverable A will be a goal definition for performance
68)     for the implementation phase. The design goal is to shrink the Tor Directory
69)     size that needs to be downloaded to about 300 Kilobytes, which would enable an user on a 14.4
70)     kbps line to download it in roughly three minutes. There may be deviations from this
71)     design goal if required to maintain anonymity and security, but this is the figure to aim for.

72)     </em></small>

73)   </td>
74)   <td>
75)     July 15, 2008
76)   </td>
77) </tr>
78) 
79) <tr>
80)   <td>
81)     <b>Deliverable B:</b>Implementation of protocol change<br />
82)     <small><em>After design, evaluation and peer review the modifications
83)     need to be implemented and integrated with the current Tor code base. The actual implementation of
84)     the necessary changes will take approximately three months.
85)     </em></small>
86)   </td>
87)   <td>
88)     October 15, 2008
89)   </td>
90) </tr>
91) 
92) <tr bgcolor="#e5e5e5">
93)   <td>
94)     <b>Deliverable C:</b>Testing<br />
95)     <small><em>Since the modification is highly critical to the security
96)     and anonymity of the Tor network, it requires extensive testing and debugging in laboratory and real life
97)     conditions. A period of three months is projected for testing and debugging, where the
98)     responsible developer is committed to the testing effort with 1/3 of its time. Part of the
99)     testing phase will be a public beta period.
100)     </em></small>
101)   </td>
102)   <td>

103)     January 15, 2009

104)   </td>
105) </tr>
106) 
107) <tr>
108)   <td>
109)     <b>Deliverable D:</b>Rollout<br />
110)     <small><em>The actual rollout to the Tor server network will be
111)       conducted in sync with the regular Tor
112)       release schedule. As this schedule is dependent on a number of external
113)       factors, like the completion of other software projects that should go into the same
114)       release, the actual release time and the time until this release has been accepted and
115)       installed by most Tor server operators can vary. From experience a period of three to four
116)       months can be expected. The rollout will be conducted as part of the regular Tor
117)       release process that is a persistent activity done by volunteers and by personal paid through
118)       other grants to the Tor project.
119)     </em></small>
120)   </td>
121)   <td>

122)     April 15, 2009

123)   </td>
124) </tr>
125) </table>
126) 
127) <br />
128) 
129) <a id="Reports"></a>
130) <h2><a class="anchor" href="#Reports">Monthly Status Reports</a></h2>
131) <p>
132) There will be in total eight monthly status reports beginning with the
133) first deliverable on July 15, 2008 and ending with completion of

134) implementation and testing work on January 15, 2009.

135) </p>
136) 

137) <table width="100%" border="0" cellspacing="0" cellpadding="3">
138) <thead>
139) <tr>
140) <th><big>Month,</big></th>
141) <th><big>Status Report</big></th>
142) </tr>
143) </thead>
144) 

145) <tr bgcolor="#e5e5e5">
146)   <td>

147)     <a id="Jun08"></a>
148)     <a class="anchor" href="#Jun08">Jun 08</a>

149)   </td>
150)   <td>
151)     <small><em>We did some initial measuring of Tor clients
152)     bootstrapping. The results are not very surprising: a client

153)     fetches about 10KB of certs, one consensus for 140KB (now 90KB, see

154)     next paragraph), and about 1.5 megs of Server Descriptors (after

155)     half of which it starts building circuits).</em></small>

156)     <br />

157)     <small><em><a href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/138-remove-down-routers-from-consensus.txt">Proposal

158)     138</a> shrinks consensus documents by 30% to
159)     40% and has already been implemented and merged into the spec.
160)     Implementation is part of the 0.2.1.x-alpha tree and the code will

161)     take effect once over two-thirds of the directory authorities

162)     (i.e. 5 out of 6) have upgraded.</em></small>

163)     <br />
164)     <small><em><a href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/140-consensus-diffs.txt">Proposal
165)     140</a> does not directly relate to reducing initial download size,
166)     but instead tries to make subsequent downloads of new consensus
167)     documents use fewer bytes has also been written up and <a
168)     href="http://archives.seul.org/or/dev/Jun-2008/msg00013.html">sent to
169)     or-dev</a>. There are some questions to be answered from other Tor
170)     developers first, but other than that I think it's fine and could

171)     be implemented.</em></small>

172)     <br />
173)     <small><em>The Big Thing is making clients not download all 1.5 megs
174)     of server descriptors. <a href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/141-jit-sd-downloads.txt">Proposal
175)     141</a> has been <a
176)     href="http://archives.seul.org/or/dev/Jun-2008/msg00017.html">sent

177)     to or-dev</a>. There are basically 3 things to it, as

178)     far as I can see at the moment: move load balancing info into the
179)     consensus (should not be hard), implement something so that Tor
180)     clients can fetch SDs on demand from routers along their circuit

181)     while they are building it (described in the draft), and deal with

182)     exit selection. We're still developing ideas for the last part;

183)     some possibilities are mentioned in the draft.</em></small>

184)   </td>
185) </tr>
186) 
187) <tr>
188)   <td>

189)     <a id="Jul08"></a>
190)     <a class="anchor" href="#Jul08">Jul 08</a>

191)   </td>
192)   <td>

193)     <small><em>Work on <a
194)     href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/141-jit-sd-downloads.txt">Proposal
195)     141</a> is continuing:  There are two basic ideas for how to move
196)     load balancing information into the consensus: One is the
197)     authorities generate the weights that clients should use and put
198)     that in the consensus, the other approach is to just put bandwidth
199)     information from the server descriptor there.  The latter option
200)     is probably friendlier when one also considers <a
201)     href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/140-consensus-diffs.txt">Proposal

202)     140</a> since it avoids a having number of highly fluctuating

203)     numbers in the consensus.</em></small>
204)     <br />
205)     <small><em>For handling exit policies <a
206)     href="http://archives.seul.org/or/dev/Jul-2008/msg00022.html">a

207)     post on the or-dev mailinglist</a> suggests that a hash identifying a

208)     node's exit policy be added to the consensus document.  The addition
209)     of another 160 high-entropy bits per node to the consensus might
210)     be a cause for concern but we think that since a lot of the exit
211)     policies are the same the consensus document will compress nicely.
212)     Measurements pending.</em></small>

213)   </td>
214) </tr>
215) 
216) <tr bgcolor="#e5e5e5">
217)   <td>

218)     <a id="Aug08"></a>
219)     <a class="anchor" href="#Aug08">Aug 08</a>

220)   </td>
221)   <td>

222)     <small><em>Directory authority voting documents, and their consensus
223)     forming algorithm have been changed to include bandwidth information
224)     and policy summaries as documented in Proposal 141.  Once five of
225)     the six running authorities have upgraded to trunk at at least
226)     revision 16554 the consensus will start to include that
227)     information.</em></small>

228)   </td>
229) </tr>
230) 
231) <tr>
232)   <td>

233)     <a id="Sep08"></a>
234)     <a class="anchor" href="#Sep08">Sep 08</a>

235)   </td>
236)   <td>

237) <small><em>No updates for September.</em></small>

238)   </td>
239) </tr>
240) 
241) <tr bgcolor="#e5e5e5">
242)   <td>

243)     <a id="Oct08"></a>
244)     <a class="anchor" href="#Oct08">Oct 08</a>

245)   </td>

246)   <td><p>
247) <small><em>We didn't hit our "finish

248) the implementation" milestone for this month since the developer leading
249) the project has too much on his plate. The good news there is that we've
250) gotten quite a bit of the implementation work done, and it's been in for
251) several months now (since the August milestone) so it's seen a lot of
252) testing already. The remaining implementation steps are to teach relays
253) how to receive requests for fetching a relay descriptor from inside a
254) circuit (we probably want to use a new cell type specifically for this,
255) so we cut out a round-trip), and then teach clients to do that when the
256) relay they're using is running a recent enough version. These two steps

257) are written in more detail in
258) <a href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/141-jit-sd-downloads.txt">Section 3.2 of proposal 141</a></em></small></p>

259) 

260) <p><small><em>Our new timing plan is to have both of these pieces in place by mid Nov,

261) and if that starts looking less likely then we're going to do a more

262) radical overhaul of our timing plan and maybe also the design.</em></small></p>

263) 

264) <p><small><em>There are several other components we'd like to get

265) to after this piece of it -- one we've been thinking about a lot lately
266) is downloading "diffs" of the latest consensus:
267) <a href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/140-consensus-diffs.txt">140-consensus-diffs.txt</a>.
268) First this could save bandwidth, which is always nice when multiplied
269) by the total number of clients, but it also means that we can use this
270) saved bandwidth to fetch the diffs more often than the current "every
271) 3 to 4 hours". If clients learn more up-to-date directory information,
272) then they can build faster paths because they have better information for
273) each relay's bandwidth (which ties into the first NLnet project above),
274) but the key new idea is that it also means that we're better at taking
275) advantage of transient relays: right now a relay needs to be up for 3 or
276) 4 hours before it sees any users, and that rules out a lot of volunteers

277) who want to run a relay but only want to be up a few hours at a time.</em></small></p>
278) <p><small><em>The next step is to get the

279) implementation for proposal 141 finished so we can start putting it in

280) front of users for testing. Soon, we hope!</em></small></p>

281)   </td>
282) </tr>
283) 
284) <tr>
285)   <td>

286)     <a id="Nov08"></a>
287)     <a class="anchor" href="#Nov08">Nov 08</a>

288)   </td>
289)   <td>

290) <p><small><em>It looks like the

291) original plan we had for the last development piece was both a) way

292) harder than we hoped, and b) hopefully overkill compared to what we need.</em></small></p>

293) 

294) <p><small><em> Roger restarted the design discussion on or-dev:
295) <a href="http://archives.seul.org/or/dev/Nov-2008/threads.html">http://archives.seul.org/or/dev/Nov-2008/threads.html</a>.</em></small></p>

296) 

297) <p><small><em>I think we now have a better handle on the options and tradeoffs:
298) <a href="http://archives.seul.org/or/dev/Nov-2008/msg00007.html">http://archives.seul.org/or/dev/Nov-2008/msg00007.html</a>.</em></small></p>

299) 

300) <p><small><em>Nick has been buried in other development projects (hopefully starting to

301) wrap up this month-ish), and I want to get his opinion on how to proceed;

302) I am hoping we'll pick one of the more simple approaches.</em></small></p>

303) 

304) <p><small><em>Alas, the really simple approaches provide less scalability. But I think

305) they will be good stopgaps until later -- and when 'later' arrives, who

306) knows what else will have changed.</em></small></p>

307)   </td>
308) </tr>
309) 
310) <tr bgcolor="#e5e5e5">
311)   <td>

312)     <a id="Jan09"></a>
313)     <a class="anchor" href="#Jan09">Jan 09</a>

314)   </td>
315)   <td>

316) <p><small><em>

317) I wrote up the more detailed version of our new design idea, as <a
318) href="https://svn.torproject.org/svn/tor/trunk/doc/spec/proposals/158-microdescriptors.txt">Tor
319) proposal #158</a>, and discussion has started at <a

320) href="http://archives.seul.org/or/dev/Jan-2009/msg00010.html">http://archives.seul.org/or/dev/Jan-2009/msg00010.html</a>.</em></small></p>

321) 

322) <p><small><em>

323) I think this is finally the one! (Well, once I finish folding in all
324) the comments.)

325) </em></small></p>

326) 

327) <p><small><em>

328) One of the big reasons why the project isn't on its original schedule is
329) that a main conclusion from <a href="<page projects/hidserv>">Karsten's
330) NLnet project on hidden service performance</a> was that it's the circuit
331) extension that's the main killer. Yet this project had proposed to add
332) more round-trips and complexity into exactly that step. So we needed
333) to build a better plan to achieve the original goal without screwing up
334) performance even more.

335) </em></small></p>

336) 

337) <p><small><em>We've been revising the design proposal over the past weeks, and

338) I think we'll be ready soon to start implementing. Note that since
339) we have a bunch of development items we're hoping to land in mid Feb
340) already, it's likely that this implementation won't land until late Feb

341) or March. This time for sure!</em></small></p>

342)   </td>
343) </tr>
344) </table>
345) 
346) <br />