git.schokokeks.org
Repositories
Help
Report an Issue
tor-webwml.git
Code
Commits
Branches
Tags
Suche
Strukturansicht:
c18f0dd52
Branches
Tags
bridges
docs-debian
jobs
master
press-clips
tor-webwml.git
trunk
about
gsocProposal
gsoc12-proposal-stemImprovements.html
Added 19 FAQ entries
Matt Pagan
commited
c18f0dd52
at 2013-08-26 04:06:05
gsoc12-proposal-stemImprovements.html
Blame
History
Raw
<head> <title>Stem Improvements and Arm port</title> <style> h1 {font-size: 20pt;} </style> </head> <body> <h1>What project would you like to work on?</h1> <p><br /><span style="font-size: small;">I would like to work on adding multiple features to Stem and on porting Arm to</span><br /><span style="font-size: small;">use Stem. I originally intended to work on implementing a PathSupport equivalent</span><br /><span style="font-size: small;">module for Stem. But, I was convinced that it wouldn't be the best idea for a</span><br /><span style="font-size: small;">gsoc project. </span><br /><br /><span style="font-size: small;">The improvements I am going to make to stem in particular are, addition of a</span><br /><span style="font-size: small;">general controller class, integrating the safe cookie authentication client</span><br /><span style="font-size: small;">into Stem and extending stem.descriptor by implementing network status descriptor</span><br /><span style="font-size: small;">and microdescriptor parsing functionality. I will also begin porting Arm to use Stem.</span></p> <h2> </h2> <h2>General Controller Class</h2> <p><span style="font-size: small;">The general controller class will provide basic functionality for accessing</span><br /><span style="font-size: small;">control port commands at a higher level (using methods instead of sending the</span><br /><span style="font-size: small;">commands to the socket). This will also (ideally) be used a base class for</span><br /><span style="font-size: small;">implementing any advanced controllers, such as a consensus tracking controller</span><br /><span style="font-size: small;">that would override the newconsensus/ns methods to keep track of the current</span><br /><span style="font-size: small;">consensus.</span><br /><br /><span style="font-size: small;">* This class will also implement other necessary helper classes/functions, such</span><br /><span style="font-size: small;"> as the Router class.</span><br /><br /><span style="font-size: small;">* This class will implements methods that can be called to send every known</span><br /><span style="font-size: small;"> control command to the control port. This includes the following commands:</span><br /><span style="font-size: small;"> SETCONF, RESETCONF, GETCONF, SETEVENTS, SAVECONF, SIGNAL, AUTHENTICATE, MAPADDRESS, GETINFO, EXTENDCIRCUIT, SETCIRCUITPURPOSE, SETROUTERPURPOSE, ATTACHSTREAM, AUTHCHALLENGE, POSTDESCRIPTOR, REDIRECTSTREAM, CLOSESTREAM, CLOSECIRCUIT, QUIT, USEFEATURE, RESOLVE, LOADCONF, PROTOCOLINFO, TAKEOWNERSHIP.</span></p> <p><span style="font-size: small;">* This package implements classes which can parse the output of the above control commands and store the parsed data. These classes will be subclasses of ControlMessage, and the corresponding class for a control command FOOBAR will be called FooBarResponse, usually. For example, the GetInfoResponse class would look something like this (minus the error checking, documentation etc.):</span></p> <p><code><samp><span style="font-size: small;">class GetInfoResponse(stem.socket.ControlMessage):<br /> @staticmethod<br /> def convert(control_message):<br /> control_message.__class__ = GetInfoResponse<br /> control_message._parse_message()<br /><br /> return control_message<br /><br /> def _parse_message(self):<br /> self.responses = {}<br /><br /> for reply in self:<br /> if reply == "OK": break<br /> elif reply.is_empty(): continue<br /><br /> k, v = reply.split("=", 1)<br /> if v[0] == "\n": # multiline reply<br /> self.responses[k] = v[1:-2] #strip \n after the = and the \n. at the end<br /> else:<br /> self.responses[k] = v<br /><br /></span></samp></code></p> <p><span style="font-size: small;">* Implement methods which are called when an asynchronous message/command is</span><br /><span style="font-size: small;"> received. The events that are received can be modified using the wrapper</span><br /><span style="font-size: small;"> around SETEVENTS. The BaseController takes care of queuing the events for</span><br /><span style="font-size: small;"> us.</span><br /><br /><span style="font-size: small;">* This package will contain exceptions for the various error codes that the control</span><br /><span style="font-size: small;"> spec defines. The general controller itself will try to handle all cases in</span><br /><span style="font-size: small;"> which an exception is raised. These exceptions will also be used by</span><br /><span style="font-size: small;"> controllers which will subclass the general controller class.</span><br /><br /><span style="font-size: small;">* This will also involve writing a lot of unit and integration tests. I plan to</span><br /><span style="font-size: small;"> write the integration tests by creating a test controller that is a subclass</span><br /><span style="font-size: small;"> of the general controller class and running the tests against it. The unit</span><br /><span style="font-size: small;"> tests will be written against an object of the general controller class and against objects of the individual *Response classes.</span><br /><br /><span style="font-size: small;">* I will also be writing comprehensive documentation that will help people write</span><br /><span style="font-size: small;"> their own controller classes which subclasses this without having to go</span><br /><span style="font-size: small;"> through the sources. This mostly involves writing concise (and sometimes elaborate) </span><span style="font-size: small;">docstrings</span><span style="font-size: small;">. I will also be writing a few (2-4) self contained examples of code that subclasses the general controller class. These will be placed in the examples directory.<br /></span></p> <h2><br />Safe Cookie Authentication</h2> <p><span style="font-size: small;">Currently, Stem does not understand the new safe cookie authentication method.</span><br /><span style="font-size: small;">There exists a python script that does the authentication.</span><br /><br /><span style="font-size: small;">This task entails distilling the authentication client script to extract the</span><br /><span style="font-size: small;">authentication specific parts, and then integrating it into the connection</span><br /><span style="font-size: small;">module while writing new code as necessary.</span><br /><br /><span style="font-size: small;">I will also be writing integration tests. These will be inspired by the current</span><br /><span style="font-size: small;">tests for cookie based authentication, if not completely mimicking them</span><br /><br /><span style="font-size: small;">This task will end with closing <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a><br /></span></p> <h2> </h2> <h2>Descriptor</h2> <p style="text-align: left;"><span style="font-size: small;">stem.descriptor is a python counterpart of metrics-lib. Currently, it is missing</span><br /><span style="font-size: small;">a lot of functionality provided by metrics-lib. At this moment, it can parse</span><br /><span style="font-size: small;">V3 server descriptors using parse_file_v3 in stem.descriptor.server_descriptor.</span><br /><br /><span style="font-size: small;">I will be extending stem.descriptor by implementing methods and classes to</span><br /><span style="font-size: small;">handle the following descriptor formats:</span><br /><br /></p> <ol> <li><span style="font-size: small;">V3 network status documents</span></li> <li><span style="font-size: small;">Microdescriptor documents</span></li> <li><span style="font-size: small;">Extra-info documents<br /></span></li> </ol> <p><br /><span style="font-size: small;">I will be implementing the parsers and classes that contain the parsed data. I</span><br /><span style="font-size: small;">will also be implementing methods that will get this data from common data</span><br /><span style="font-size: small;">sources.</span><br /><br /><span style="font-size: small;">I will also be writing unit and integration tests to ensure the parsers are</span><br /><span style="font-size: small;">working as expected. While this task will require writing a large number of</span><br /><span style="font-size: small;">tests, documentation will be a relatively simple subtask and won't require as</span><br /><span style="font-size: small;">much time as the documentation of the general controller class.</span></p> <h2><br />Arm Port</h2> <p><span style="font-size: small;">Porting an application to use Stem is an exercise that will test its design.</span><br /><span style="font-size: small;">Arm is the Anonymous Relay Monitor. I will port Arm to use Stem instead of</span><br /><span style="font-size: small;">TorCtl.</span><br /><br /><span style="font-size: small;">This requires the general controller class to be implemented. I assume that</span><br /><span style="font-size: small;">I will make atleast some minor modifications to the Stem API during this</span><br /><span style="font-size: small;">process.</span><br /><br /><span style="font-size: small;">Though at first, this appears like it is a very large task, the time to</span><br /><span style="font-size: small;">implement is reduced greatly because many things that Arm does on its own are</span><br /><span style="font-size: small;">now done by Stem. Ex: version parsing, connection handling, get_pid methods etc.</span><br /><br /><span style="font-size: small;">The newly implemented general controller class will also help greatly in</span><br /><span style="font-size: small;">reducing the amount of time taken to port this application, since the Controller</span><br /><span style="font-size: small;">class forms a significant chunk of the code that needs to be ported.</span></p> <p><span style="font-size: small;">Almost all of the work that needs to be here is porting a single file - src/util/torTools.py which also contains the controller class. </span><span style="font-size: small;">This file currently consist of a few functions that are already implemented in Stem (the version parsing, get_pid mentioned above). </span><span style="font-size: small;">I will begin with replacing the connection handling code and the miscellaneous functions defined in this file. </span><span style="font-size: small;"> The code that requires TorCtl within the controller will be substituted with the appropriate Stem code. (At a later date, '</span><span style="font-size: small;">outside' of gsoc, </span><span style="font-size: small;">this will be refactored to use Stem better).</span></p> <p><span style="font-size: small;">There is almost negligible code outside of this single file that imports TorCtl and uses it directly. I will refactor these files such that they make these calls via torTools.py so that the controller library is fully abstracted in that single file.<br /></span></p> <h2><br />Deliverables</h2> <h3><br />Mid term evaluation</h3> <p><span style="font-size: small;">* Implementation of a fully documented General controller class with sufficient</span><br /><span style="font-size: small;"> test coverage.</span><br /><span style="font-size: small;">* Implementation of parsers for parsing network status documents and</span><br /><span style="font-size: small;"> microdescriptor documents with significant test coverage.</span><br /><span style="font-size: small;">* Integrate the Safe Cookie authentication client into Stem. (Close <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a>).</span></p> <h3> </h3> <h3>Final Evaluation</h3> <p><span style="font-size: small;">* Implementation of extra-info document parser. </span></p> <p><span style="font-size: small;">* A port of Arm which uses Stem. Ideally I will merge my stem port with the</span><br /><span style="font-size: small;"> master branch to completely remove the TorCtl dependency and make the next</span><br /><span style="font-size: small;"> Arm release Stem-based </span></p> <h2><br />Timeline</h2> <h3><br />April 23rd - May 20th Community bonding period</h3> <p><span style="font-size: small;">I'll spend this time working on minor stem bugs. I will also be</span><span style="font-size: small;"> familiarizing myself with the Arm codebase.</span></p> <p> </p> <h3>May 20th - July 9th Coding Period (Pre-mid term evaluation)</h3> <p><span style="font-size: small;">Week 1</span></p> <p><span style="font-size: small;">Integrate safe cookie authentication into Stem. Write integration tests. Close <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a></span></p> <p><span style="font-size: small;">Implement the classes required to for the general controller class and other</span><br /><span style="font-size: small;">helper functions (Router, Exceptions, etc.).</span><br /><br /><span style="font-size: small;">Week 2</span><br /><br /><span style="font-size: small;">Implement wrapper classes for AUTHENTICATE and PROTOCOLINFO. Implement the</span><br /><span style="font-size: small;">following control command parser/container classes for the following commands:</span><br /><br /><span style="font-size: small;">* GETINFO</span><br /><span style="font-size: small;">* GETCONF</span><br /><span style="font-size: small;">* SETCONF</span><br /><span style="font-size: small;">* LOADCONF</span><br /><span style="font-size: small;">* RESETCONF</span><br /><span style="font-size: small;">* SAVECONF</span><br /><span style="font-size: small;">* SIGNAL</span><br /><span style="font-size: small;">* TAKEOWNERSHIP</span><br /><span style="font-size: small;">* USEFEATURE</span><br /><span style="font-size: small;">* QUIT</span><br /><br /><span style="font-size: small;">Week 3</span><br /><br /><span style="font-size: small;">Implement control command parser/container for the following commands:</span><br /><br /><span style="font-size: small;">* EXTENDCIRCUIT</span><br /><span style="font-size: small;">* SETCIRCUITPURPOSE</span><br /><span style="font-size: small;">* ATTACHSTREAM</span><br /><span style="font-size: small;">* REDIRECTSTREAM</span><br /><span style="font-size: small;">* CLOSESTREAM</span><br /><span style="font-size: small;">* CLOSECIRCUIT</span><br /><span style="font-size: small;">* MAPADDRESS</span><br /><span style="font-size: small;">* RESOLVE</span><br /><span style="font-size: small;">* POSTDESCRIPTOR</span><br /><br /><span style="font-size: small;">Week 4</span><br /><br /><span style="font-size: small;">Implement control command parser/container for the SETEVENTS command.</span><br /><span style="font-size: small;">Implement the event handling methods for the following asynchronous events:</span><br /><br /><span style="font-size: small;">* Log messages (DEBUG/INFO/NOTICE/WARN/ERR)</span><br /><span style="font-size: small;">* CIRC</span><br /><span style="font-size: small;">* STREAM</span><br /><span style="font-size: small;">* OR</span><br /><span style="font-size: small;">* BW</span><br /><span style="font-size: small;">* SIGNAL</span><br /><span style="font-size: small;"> * STREAM_BW</span><br /><br /><span style="font-size: small;">Week 5</span><br /><br /><span style="font-size: small;">Implement the event handling methods for the following asynchronous events:</span><br /><br /><span style="font-size: small;"> * NEWDESC</span><br /><span style="font-size: small;">* DESCCHANGED</span><br /><span style="font-size: small;">* ADDRMAP</span><br /><span style="font-size: small;">* Status reports</span><br /><span style="font-size: small;"> * GUARD</span><br /><span style="font-size: small;"> * BUILDTIMEOUT_SET</span><br /><span style="font-size: small;"> * CIRC_MINOR</span><br /><span style="font-size: small;"> * CLIENTS_SEEN</span><br /><br /><span style="font-size: small;">Week 6</span><br /><br /><span style="font-size: small;">Implement the event handling methods for the following asynchronous events:</span><br /><br /><span style="font-size: small;">* NS</span><br /><span style="font-size: small;">* NEWCONSENSUS</span><br /><br /><span style="font-size: small;">Writing additional documentation for the controller class. Writing any</span><br /><span style="font-size: small;">additional tests. Buffer time.</span><br /><br /><span style="font-size: small;">Week 7</span><br /><br /><span style="font-size: small;">Implement a parser for parsing network status descriptors and the</span><br /><span style="font-size: small;">microdescriptors. Write integration tests for the parser. Implement classes for</span><br /><span style="font-size: small;">storing this parsed descriptor data. Write functions to get them from common</span><br /><span style="font-size: small;">data sources.</span></p> <p><span style="font-size: small;">Week 8</span></p> <p><span style="font-size: small;">Additional buffer time to complete any pending controller class work. (Otherwise, begin work on the extra-info document parser).</span></p> <h3>July 9th - August 13th (Post-mid term evaluation)</h3> <p><span style="font-size: small;">Week 9<br /></span></p> <p><span style="font-size: small;">Implement a parser for parsing extra-info documents.</span><span style="font-size: small;"><br /></span></p> <p> </p> <p><span style="font-size: small;">Week 10</span><br /><br /><span style="font-size: small;">Begin porting Arm to use Stem. Start with replacing the functions in torTools.py with their stem equivalents. Begin replacing TorCtl code in the controller class with the equivalent Stem code.<br /></span></p> <p><br /><span style="font-size: small;">Week 11-12</span><br /><br /><span style="font-size: small;">More controller class work. Replace the little TorCtl elsewhere in the codebase </span><span style="font-size: small;">with equivalent Stem code. I will be left with some buffer time here.<br /></span></p> <h3><br />August 13th - August 20th (Post-soft pencils down deadline)</h3> <p><span style="font-size: small;">Week 13</span><br /><br /><span style="font-size: small;">Post soft-pencils down week. Buffer time to finish any unfinished work. Any</span> <span style="font-size: small;">extra time will be spent on documenting Stem.</span></p> <h1> </h1> <h1>Point us to a code sample: something good and clean to demonstrate that you know what you're doing, ideally from an existing project.</h1> <p><span style="font-size: small;">I have written a few patches for some Tor Project projects, <a title="#1667" href="https://trac.torproject.org/projects/tor/ticket/1667">#1667</a> (Tor), <a title="#5032" href="https://trac.torproject.org/projects/tor/ticket/5032">#5032</a> (Thandy). Two to Stem, which have been committed to the repository <a title="#5199" href="https://trac.torproject.org/projects/tor/ticket/5199">#5199</a> and <a title="#5472" href="https://trac.torproject.org/projects/tor/ticket/5472">#5472</a>.</span></p> <h1> </h1> <h1>Why do you want to work with The Tor Project / EFF in particular?</h1> <p><br /><span style="font-size: small;">I began reading stuff about The Tor Project about 2 months ago after</span><br /><span style="font-size: small;">Sathyanarayanan suggested that I contribute to it.</span><br /><br /><span style="font-size: small;">Now, I love the internet, and it is responsible for a large part who I am. The</span><br /><span style="font-size: small;">Tor Project and the EFF work to defend the things that make the internet what it</span><br /><span style="font-size: small;">is, i.e. (among other things) free speech.</span><br /><br /><span style="font-size: small;">I can relate with this goal, and this is why I want to work with The Tor Project/EFF.</span></p> <h1><br />Tell us about your experiences in free software development environments. We especially want to hear examples of how you have collaborated with others rather than just working on a project by yourself.</h1> <p><br /><span style="font-size: small;">Though I have been using Free software for a long time (I switched to Linux</span><br /><span style="font-size: small;">about 7 years ago), I haven't made any significant contributions to free</span><br /><span style="font-size: small;">software, apart from a few bugs reports and minor patches. However, I am</span><br /><span style="font-size: small;">familiar with version control software, bug trackers etc. I have used them while</span><br /><span style="font-size: small;">submitting the patches mentioned earlier.</span></p> <h1> </h1> <h1><br />Will you be working full-time on the project for the summer, or will you have other commitments too (a second job, classes, etc)? If you won't be available full-time, please explain, and list timing if you know them for other major deadlines (e.g. exams). Having other activities isn't a deal-breaker, but we don't want to be surprised.</h1> <p> </p> <p><span style="font-size: small;">I have exams until the 29th of April, so I will be missing a few days of the</span><br /><span style="font-size: small;">community bonding period, though, I hope to show up on the IRC channels even</span><br /><span style="font-size: small;">then, albeit sporadically. I also might have to write an exam either in july or</span><br /><span style="font-size: small;">august. Though, that depends on me flunking. It won't cost me more than 2 days,</span><br /><span style="font-size: small;">and I will work extra during the weekends to make up for it.</span></p> <h1><br />Will your project need more work and/or maintenance after the summer ends? What are the chances you will stick around and help out with that and other related projects?</h1> <p><br /><span style="font-size: small;">Stem, like all libraries implementing an API for a moving target requires</span><br /><span style="font-size: small;">maintenance. I will co-maintain Stem in the future. By the time I'm done with</span><br /><span style="font-size: small;">the SoC program. I would've also gained familiarity with Arm. I'll be able to</span><br /><span style="font-size: small;">contribute to it, and I will.</span><br /><br /><span style="font-size: small;">Personally, I am also interested in getting involved in Tor development, and</span><br /><span style="font-size: small;">the re-implementation of Thandy (if/when it happens).</span></p> <h1><br />What is your ideal approach to keeping everybody informed of your progress, problems, and questions over the course of the project? Said another way, how much of a "manager" will you need your mentor to be?</h1> <p><br /><span style="font-size: small;">IRC is my preferred mode of communication, and I will be using it to ask</span><br /><span style="font-size: small;">questions and for help with my problems. If I'm unable to get the answer I want</span><br /><span style="font-size: small;">on the IRC, I will ask them on the mailing list.</span><br /><br /><span style="font-size: small;">I will keep people informed about my progress by sending (probably monthly, or</span><br /><span style="font-size: small;">as often as required) reports the mailing list.</span></p> <h1> </h1> <h1>What school are you attending? What year are you, and what's your major/degree/focus? If you're part of a research group, which one?</h1> <p><br /><span style="font-size: small;">I'm an undergraduate student majoring in computer science studying at GITAM</span><br /><span style="font-size: small;">University I'm currently working on my final year project which involves</span><br /><span style="font-size: small;">computer network modelling.</span></p> <h1> </h1> <h1><br />How can we contact you to ask you further questions? Google doesn't share your contact details with us automatically, so you should include that in your application. In addition, what's your IRC nickname? Interacting with us on IRC will help us get to know you, and help you get to know our community.</h1> <p><br /><span style="font-size: small;">I'm available via email at <removed>. I'm also subscribed to many</span><br /><span style="font-size: small;">of the tor-* mailing lists, including tor-dev, tor-talk, tor-bugs and tor-commits. My nickname on OFTC</span><br /><span style="font-size: small;">is 'neena'. My email account also doubles up as my Jabber account, though, I</span><br /><span style="font-size: small;">prefer IRC.</span><br /><span style="font-size: small;"> </span></p> <h1><br />Are you applying to other projects for GSoC and, if so, what would be your preference if you're accepted to both? Having a stated preference helps with the deduplication process and will not impact if we accept your application or not.</h1> <p><br /><span style="font-size: small;">I am not applying to any other projects for GSoC.</span><br /><br /></p> <p> </p> </body>
