Adding a project to expand stem's tests to more dedicatedly test tor
Damian Johnson commited on 2013-03-09 07:41:12
Zeige 1 geänderte Dateien mit 33 Einfügungen und 1 Löschungen.
This is a project that I have been batting around with Nick for a bit. It doesn't include the 'grand vision' he wants for covering inter-network components, but it would provide dedicated tests for a very substantial portion of Tor's codebase. While I think a more complicated network simulator and testing suite would yield greater benefits, I also think it would be *vastly* harder to make and hence quite likely to never happen. Presently our automated testing of Tor is pretty poor and there's no need to let the perfect be the enemy of the good on this.
- getinvolved/en/volunteer.wml 53536194c..54739af0b
| ... | ... |
@@ -660,7 +660,8 @@ meetings around the world.</li> |
| 660 | 660 |
|
| 661 | 661 |
<p> |
| 662 | 662 |
<b>Project Ideas:</b><br /> |
| 663 |
- <i><a href="#stemUsability">Stem Usability and Porting</a></i> |
|
| 663 |
+ <i><a href="#stemUsability">Stem Usability and Porting</a></i><br /> |
|
| 664 |
+ <i><a href="#stemTestingForTor">Stem Tests for Tor</a></i> |
|
| 664 | 665 |
</p> |
| 665 | 666 |
|
| 666 | 667 |
<a id="project-txtorcon"></a> |
| ... | ... |
@@ -1694,6 +1695,37 @@ meetings around the world.</li> |
| 1694 | 1695 |
<b>As part of your application for this project please write a script that does something interesting with stem.</b> Bonus points if this is something that we can <a href="https://stem.torproject.org/tutorial.html">make a tutorial</a> around! |
| 1695 | 1696 |
</p> |
| 1696 | 1697 |
|
| 1698 |
+ <a id="stemTestingForTor"></a> |
|
| 1699 |
+ <li> |
|
| 1700 |
+ <b>Stem Tests for Tor</b> |
|
| 1701 |
+ <br> |
|
| 1702 |
+ Effort Level: <i>Medium</i> |
|
| 1703 |
+ <br> |
|
| 1704 |
+ Skill Level: <i>Medium</i> |
|
| 1705 |
+ <br> |
|
| 1706 |
+ Likely Mentors: <i>Damian (atagar)</i> |
|
| 1707 |
+ <p> |
|
| 1708 |
+ Stem is a library for interacting with Tor (see '<a href="#stemUsability">Stem Usability and Porting</a>' above for a summary). The library has both <a href="https://gitweb.torproject.org/stem.git/tree/HEAD:/test/unit">unit</a> and <a href="https://gitweb.torproject.org/stem.git/tree/HEAD:/test/integ">integration</a> tests. The unit tests provide a quick, direct test of stem's codebase while the integration test exercises its functionality against a live instance of Tor. |
|
| 1709 |
+ </p> |
|
| 1710 |
+ |
|
| 1711 |
+ <p> |
|
| 1712 |
+ Stem's integration tests have thus far been (unsurprisingly) designed to test stem but there's no need for them to be limited to that. Stem is a complete implementation of Tor's <a href="https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt">control-spec</a> and <a href="https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt">dir-spec</a>. As such, stem's tests could be easily expanded to more dedicatedly test behavior involved in those portions of Tor's codebase, as well as provide a smoke test for its general functionality. |
|
| 1713 |
+ </p> |
|
| 1714 |
+ |
|
| 1715 |
+ <p> |
|
| 1716 |
+ This project would involve several components: |
|
| 1717 |
+ </p> |
|
| 1718 |
+ |
|
| 1719 |
+ <ol> |
|
| 1720 |
+ <li>Determine what kind of tests we need. <b>This should be done during the application phase</b> by <a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev/">contacting tor-dev@</a>. Hopefully this will give us an idea of what would be the most useful kind of tests of this nature for Tor development.</li> |
|
| 1721 |
+ <li>To be useful our integration tests need to continually run against the present tip of Tor's codebase. To do this we'll want to (1) fetch and compile the latest version of Tor, (2) run our integration tests against it, (3) compose the results as an html formatted email, and send it to a list. (<a href="https://trac.torproject.org/8261">ticket</a>)</li> |
|
| 1722 |
+ <li>Implement the new suite of integration tests for Tor. This will likely include expanding Tor to support better testability. One useful candidate, for instance, would be a controller method to fetch our own descriptor. This would let us easily test various configurations to see if they provide valid descriptor content.</li> |
|
| 1723 |
+ </ol> |
|
| 1724 |
+ |
|
| 1725 |
+ <p> |
|
| 1726 |
+ <b>As part of your application for this project please write some code to expand stem's tests.</b> Bonus points if it impelments one of your suggestions for better testing Tor! |
|
| 1727 |
+ </p> |
|
| 1728 |
+ |
|
| 1697 | 1729 |
<!-- |
| 1698 | 1730 |
<a id="tailsServer"></a> |
| 1699 | 1731 |
<li> |
| 1700 | 1732 |