<head>
<title>Stem Improvements and Arm port</title>
<style>
h1 {font-size: 20pt;}
</style>
</head>
<body>
<h1>What project would you like to work on?</h1>
<p><br /><span style="font-size: small;">I would like to work on adding multiple features to Stem and on porting Arm to</span><br /><span style="font-size: small;">use Stem. I originally intended to work on implementing a PathSupport equivalent</span><br /><span style="font-size: small;">module for Stem. But, I was convinced that it wouldn't be the best idea for a</span><br /><span style="font-size: small;">gsoc project. </span><br /><br /><span style="font-size: small;">The improvements I am going to make to stem in particular are, addition of a</span><br /><span style="font-size: small;">general controller class, integrating the safe cookie authentication client</span><br /><span style="font-size: small;">into Stem and extending stem.descriptor by implementing network status descriptor</span><br /><span style="font-size: small;">and microdescriptor parsing functionality. I will also begin porting Arm to use Stem.</span></p>
<h2> </h2>
<h2>General Controller Class</h2>
<p><span style="font-size: small;">The general controller class will provide basic functionality for accessing</span><br /><span style="font-size: small;">control port commands at a higher level (using methods instead of sending the</span><br /><span style="font-size: small;">commands to the socket). This will also (ideally) be used a base class for</span><br /><span style="font-size: small;">implementing any advanced controllers, such as a consensus tracking controller</span><br /><span style="font-size: small;">that would override the newconsensus/ns methods to keep track of the current</span><br /><span style="font-size: small;">consensus.</span><br /><br /><span style="font-size: small;">* This class will also implement other necessary helper classes/functions, such</span><br /><span style="font-size: small;"> as the Router class.</span><br /><br /><span style="font-size: small;">* This class will implements methods that can be called to send every known</span><br /><span style="font-size: small;"> control command to the control port. This includes the following commands:</span><br /><span style="font-size: small;"> SETCONF, RESETCONF, GETCONF, SETEVENTS, SAVECONF, SIGNAL, AUTHENTICATE, MAPADDRESS, GETINFO, EXTENDCIRCUIT, SETCIRCUITPURPOSE, SETROUTERPURPOSE, ATTACHSTREAM, AUTHCHALLENGE, POSTDESCRIPTOR, REDIRECTSTREAM, CLOSESTREAM, CLOSECIRCUIT, QUIT, USEFEATURE, RESOLVE, LOADCONF, PROTOCOLINFO, TAKEOWNERSHIP.</span></p>
<p><span style="font-size: small;">* This package implements classes which can parse the output of the above control commands and store the parsed data. These classes will be subclasses of ControlMessage, and the corresponding class for a control command FOOBAR will be called FooBarResponse, usually. For example, the GetInfoResponse class would look something like this (minus the error checking, documentation etc.):</span></p>
<p><code><samp><span style="font-size: small;">class GetInfoResponse(stem.socket.ControlMessage):<br /> @staticmethod<br /> def convert(control_message):<br /> control_message.__class__ = GetInfoResponse<br /> control_message._parse_message()<br /><br /> return control_message<br /><br /> def _parse_message(self):<br /> self.responses = {}<br /><br /> for reply in self:<br /> if reply == "OK": break<br /> elif reply.is_empty(): continue<br /><br /> k, v = reply.split("=", 1)<br /> if v[0] == "\n": # multiline reply<br /> self.responses[k] = v[1:-2] #strip \n after the = and the \n. at the end<br /> else:<br /> self.responses[k] = v<br /><br /></span></samp></code></p>
<p><span style="font-size: small;">* Implement methods which are called when an asynchronous message/command is</span><br /><span style="font-size: small;"> received. The events that are received can be modified using the wrapper</span><br /><span style="font-size: small;"> around SETEVENTS. The BaseController takes care of queuing the events for</span><br /><span style="font-size: small;"> us.</span><br /><br /><span style="font-size: small;">* This package will contain exceptions for the various error codes that the control</span><br /><span style="font-size: small;"> spec defines. The general controller itself will try to handle all cases in</span><br /><span style="font-size: small;"> which an exception is raised. These exceptions will also be used by</span><br /><span style="font-size: small;"> controllers which will subclass the general controller class.</span><br /><br /><span style="font-size: small;">* This will also involve writing a lot of unit and integration tests. I plan to</span><br /><span style="font-size: small;"> write the integration tests by creating a test controller that is a subclass</span><br /><span style="font-size: small;"> of the general controller class and running the tests against it. The unit</span><br /><span style="font-size: small;"> tests will be written against an object of the general controller class and against objects of the individual *Response classes.</span><br /><br /><span style="font-size: small;">* I will also be writing comprehensive documentation that will help people write</span><br /><span style="font-size: small;"> their own controller classes which subclasses this without having to go</span><br /><span style="font-size: small;"> through the sources. This mostly involves writing concise (and sometimes elaborate) </span><span style="font-size: small;">docstrings</span><span style="font-size: small;">. I will also be writing a few (2-4) self contained examples of code that subclasses the general controller class. These will be placed in the examples directory.<br /></span></p>
<h2><br />Safe Cookie Authentication</h2>
<p><span style="font-size: small;">Currently, Stem does not understand the new safe cookie authentication method.</span><br /><span style="font-size: small;">There exists a python script that does the authentication.</span><br /><br /><span style="font-size: small;">This task entails distilling the authentication client script to extract the</span><br /><span style="font-size: small;">authentication specific parts, and then integrating it into the connection</span><br /><span style="font-size: small;">module while writing new code as necessary.</span><br /><br /><span style="font-size: small;">I will also be writing integration tests. These will be inspired by the current</span><br /><span style="font-size: small;">tests for cookie based authentication, if not completely mimicking them</span><br /><br /><span style="font-size: small;">This task will end with closing <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a><br /></span></p>
<h2> </h2>
<h2>Descriptor</h2>
<p style="text-align: left;"><span style="font-size: small;">stem.descriptor is a python counterpart of metrics-lib. Currently, it is missing</span><br /><span style="font-size: small;">a lot of functionality provided by metrics-lib. At this moment, it can parse</span><br /><span style="font-size: small;">V3 server descriptors using parse_file_v3 in stem.descriptor.server_descriptor.</span><br /><br /><span style="font-size: small;">I will be extending stem.descriptor by implementing methods and classes to</span><br /><span style="font-size: small;">handle the following descriptor formats:</span><br /><br /></p>
<ol>
<li><span style="font-size: small;">V3 network status documents</span></li>
<li><span style="font-size: small;">Microdescriptor documents</span></li>
<li><span style="font-size: small;">Extra-info documents<br /></span></li>
</ol>
<p><br /><span style="font-size: small;">I will be implementing the parsers and classes that contain the parsed data. I</span><br /><span style="font-size: small;">will also be implementing methods that will get this data from common data</span><br /><span style="font-size: small;">sources.</span><br /><br /><span style="font-size: small;">I will also be writing unit and integration tests to ensure the parsers are</span><br /><span style="font-size: small;">working as expected. While this task will require writing a large number of</span><br /><span style="font-size: small;">tests, documentation will be a relatively simple subtask and won't require as</span><br /><span style="font-size: small;">much time as the documentation of the general controller class.</span></p>
<h2><br />Arm Port</h2>
<p><span style="font-size: small;">Porting an application to use Stem is an exercise that will test its design.</span><br /><span style="font-size: small;">Arm is the Anonymous Relay Monitor. I will port Arm to use Stem instead of</span><br /><span style="font-size: small;">TorCtl.</span><br /><br /><span style="font-size: small;">This requires the general controller class to be implemented. I assume that</span><br /><span style="font-size: small;">I will make atleast some minor modifications to the Stem API during this</span><br /><span style="font-size: small;">process.</span><br /><br /><span style="font-size: small;">Though at first, this appears like it is a very large task, the time to</span><br /><span style="font-size: small;">implement is reduced greatly because many things that Arm does on its own are</span><br /><span style="font-size: small;">now done by Stem. Ex: version parsing, connection handling, get_pid methods etc.</span><br /><br /><span style="font-size: small;">The newly implemented general controller class will also help greatly in</span><br /><span style="font-size: small;">reducing the amount of time taken to port this application, since the Controller</span><br /><span style="font-size: small;">class forms a significant chunk of the code that needs to be ported.</span></p>
<p><span style="font-size: small;">Almost all of the work that needs to be here is porting a single file - src/util/torTools.py which also contains the controller class. </span><span style="font-size: small;">This file currently consist of a few functions that are already implemented in Stem (the version parsing, get_pid mentioned above). </span><span style="font-size: small;">I will begin with replacing the connection handling code and the miscellaneous functions defined in this file. </span><span style="font-size: small;"> The code that requires TorCtl within the controller will be substituted with the appropriate Stem code. (At a later date, '</span><span style="font-size: small;">outside' of gsoc, </span><span style="font-size: small;">this will be refactored to use Stem better).</span></p>
<p><span style="font-size: small;">There is almost negligible code outside of this single file that imports TorCtl and uses it directly. I will refactor these files such that they make these calls via torTools.py so that the controller library is fully abstracted in that single file.<br /></span></p>
<h2><br />Deliverables</h2>
<h3><br />Mid term evaluation</h3>
<p><span style="font-size: small;">* Implementation of a fully documented General controller class with sufficient</span><br /><span style="font-size: small;"> test coverage.</span><br /><span style="font-size: small;">* Implementation of parsers for parsing network status documents and</span><br /><span style="font-size: small;"> microdescriptor documents with significant test coverage.</span><br /><span style="font-size: small;">* Integrate the Safe Cookie authentication client into Stem. (Close <a href="https://trac.torproject.org/projects/tor/ticket/5262">#5262</a>).</span></p>
<h3> </h3>
<h3>Final Evaluation</h3>
<p><span style="font-size: small;">* Implementation of extra-info document parser. </span></p>
<p><span style="font-size: small;">* A port of Arm which uses Stem. Ideally I will merge my stem port with the</span><br /><span style="font-size: small;"> master branch to completely remove the TorCtl dependency and make the next</span><br /><span style="font-size: small;"> Arm release Stem-based </span></p>
<h2><br />Timeline</h2>
<h3><br />April 23rd - May 20th Community bonding period</h3>
<p><span style="font-size: small;">I'll spend this time working on minor stem bugs. I will also be</span><span style="font-size: small;"> familiarizing myself with the Arm codebase.</span></p>
<p> </p>