blog.danclaudiupophttp://blog.danclaudiupop.com/2015-06-01T00:00:00+03:00Testathon experience - Cluj-Napoca, 20152015-06-01T00:00:00+03:00danclaudiupoptag:blog.danclaudiupop.com,2015-06-01:testathon/<p>Following the popular Hackathon format, testers from all around the world have
expressed their intention to organize a similar competition, where, in theory
at least, the best engineers would have the opportunity to test new
applications, and discover all kinds of bugs.</p>
<p>There are prizes of course, for best security bug, or best report, or even best
QA engineer.</p>
<p>I was looking forward to being involved in this competition since this year it
was being held in Cluj-Napoca as well.</p>
<p>Overall, I had appreciated the intention and the scope of the Testathon Cluj
2015 edition. However, there are a few aspects which made this experience a
little disappointing for me:</p>
<ul class="simple">
<li>The quantity was very important. Not that this necessarily a bad thing, but I
would have liked to see more bold approaches and not rely solely on manual
testing (even if we were testing on mobile devices).</li>
<li>People were a little agitated, and I believe they have not fully captured the
essence of this competition. For me, as I saw it, it was more about working
together as a team (testers from different companies and even different
cities) to make the application under test better. What I felt instead was
that each of the participants was eager to prove they’re better than the
rest, and while I agree that a little competition did not hurt anyone, I
would have liked to see more of that passion in testing the app, and not
winning some prize.</li>
<li>It was unclear how the prizes were decided. More exactly, I believe that a
security bug which crashed the servers and rendered the application unusable
should have received the proper praises. Instead, this defect was almost
ignored (at least when it came to awarding the title for “Best security
bug”). To be honest, I was expecting to win. I've decompiled the apk, run a
python script to search for sensitive data (grep would suffice), found their
API calls, no authentication, authorization nor throttling whatsoever. Used
locust.io and testing session was over for all participants. They gave me a
prize, but after the big winners were annoubced and since I had to attend to
wedding I had to leave earlier. No worries, they gave the prize (an ipod) to
someone else :)</li>
</ul>
<p>At the end of the day, I was happy that the testing community from Cluj-Napoca
is getting more visibility worldwide. I met some cool people at the event.</p>
Beginners guide to functionally test an API in Django2014-03-06T00:00:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2014-03-06:guide-to-functionally-test-an-api-in-django/<p>To set up our django development environment, we'll start by creating a
virtual environment. Execute the following commands in the terminal:</p>
<div class="highlight"><pre><span class="err">$</span> <span class="n">virtualenv</span> <span class="n">temp</span>
<span class="err">$</span> <span class="n">source</span> <span class="nb">bin</span><span class="o">/</span><span class="n">activate</span>
</pre></div>
<p>I've created a repo with a dummy app which exposes an api that contains a list
of musicians with albums, to serve as our testing ground. I used django rest
framework, dig into their <a class="reference external" href="http://www.django-rest-framework.org/">documentation</a> if you want to build API's for your
application. <a class="reference external" href="https://github.com/danclaudiupop/beginners-guide-to-functionally-test-an-api-in-django">Clone the repository</a>
and then run <cite>pip install -r requirements.txt</cite> to install all dependencies.
With all of the dependencies set up, we're ready to move on and start the local
web server. You should be able to type <cite>fab runserver</cite> and browse to
<a class="reference external" href="http://127.0.0.1:8000/">http://127.0.0.1:8000/</a>. You should see a list a musicians.</p>
<p>Django test client is a good approach when it comes to these types of tests.
Let's create a simple test:</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">django.test</span> <span class="kn">import</span> <span class="n">TestCase</span>
<span class="kn">from</span> <span class="nn">django.test.client</span> <span class="kn">import</span> <span class="n">Client</span>
<span class="kn">from</span> <span class="nn">django.core.urlresolvers</span> <span class="kn">import</span> <span class="n">reverse</span>
<span class="k">class</span> <span class="nc">EndpointsTestCase</span><span class="p">(</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_musician_with_albums_list_endpoint</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">reverse</span><span class="p">(</span><span class="s">'musician-list'</span><span class="p">)</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">Client</span><span class="p">()</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">print</span> <span class="n">response</span><span class="o">.</span><span class="n">content</span>
</pre></div>
<p>As we can see, no data is exposed when going to <cite>/musicians/</cite>. Let's create
a musician with some albums in <cite>setUp</cite>:</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">django.test</span> <span class="kn">import</span> <span class="n">TestCase</span>
<span class="kn">from</span> <span class="nn">django.test.client</span> <span class="kn">import</span> <span class="n">Client</span>
<span class="kn">from</span> <span class="nn">django.core.urlresolvers</span> <span class="kn">import</span> <span class="n">reverse</span>
<span class="k">class</span> <span class="nc">EndpointsTestCase</span><span class="p">(</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">test_musician</span> <span class="o">=</span> <span class="n">Musician</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">first_name</span><span class="o">=</span><span class="s">'Foo'</span><span class="p">,</span>
<span class="n">last</span><span class="o">=</span><span class="n">name</span><span class="o">=</span><span class="s">'Bar'</span><span class="p">,</span>
<span class="n">instrument</span><span class="o">=</span><span class="s">'guitar'</span>
<span class="p">)</span>
<span class="n">album1</span> <span class="o">=</span> <span class="n">Album</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">musician</span><span class="o">=</span><span class="n">test_musician</span><span class="p">,</span>
<span class="n">name</span><span class="o">=</span><span class="s">'Album1'</span><span class="p">,</span>
<span class="n">release_date</span><span class="o">=</span><span class="s">'2011-01-01'</span><span class="p">,</span>
<span class="n">num_stars</span><span class="o">=</span><span class="mi">5</span>
<span class="p">)</span>
<span class="n">album2</span> <span class="o">=</span> <span class="n">Album</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">musician</span><span class="o">=</span><span class="n">test_musician</span><span class="p">,</span>
<span class="n">name</span><span class="o">=</span><span class="s">'Album2'</span><span class="p">,</span>
<span class="n">release_date</span><span class="o">=</span><span class="s">'2012-02-02'</span><span class="p">,</span>
<span class="n">num_stars</span><span class="o">=</span><span class="mi">4</span>
<span class="p">)</span>
<span class="k">def</span> <span class="nf">test_musician_with_albums_list_endpoint</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">reverse</span><span class="p">(</span><span class="s">'musician-list'</span><span class="p">)</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">Client</span><span class="p">()</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">print</span> <span class="n">response</span><span class="o">.</span><span class="n">content</span>
</pre></div>
<p>Now we have some data displayed in api. We can start to make some assertions.
But when we fetch the response body, it returns a string. We decode the json
document using <cite>json.loads</cite>. Create a helper function fetch_url which will
return a dictionary. We should now be able to loop through the results. The
code should look like this:</p>
<div class="highlight"><pre><span class="k">def</span> <span class="nf">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="n">client</span> <span class="o">=</span> <span class="n">Client</span><span class="p">()</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="k">return</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">content</span><span class="p">)</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">test_musician_with_albums_list_endpoint</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">reverse</span><span class="p">(</span><span class="s">'musician-list'</span><span class="p">)</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">fetch_url</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">response</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s">'albums'</span><span class="p">]),</span> <span class="mi">3</span><span class="p">)</span>
</pre></div>
<p>Before we move on, let's take a look at the models. We will need to create
programmatically objects in database for testing when querying the endpoints.
You can use <a class="reference external" href="https://factoryboy.readthedocs.org/en/latest/">factory_boy</a>
library (fixtures replacement), or any other library for that matter, but we'll
stick with a more raw approach, yet powerful enough to cover our needs. Back to
models relationships, everything looks straightforward, we can see the album
model has a field called musician, which is a foreign key to the musician
model. Let's create factory.py and map the models relationship into a factory
design:</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">albumreview.models</span> <span class="kn">import</span> <span class="n">Musician</span><span class="p">,</span> <span class="n">Album</span>
<span class="k">class</span> <span class="nc">MusicianFactory</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">first_name</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">last_name</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">instrument</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="k">if</span> <span class="n">first_name</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">first_name</span> <span class="o">=</span> <span class="s">'Foo</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span>
<span class="k">if</span> <span class="n">last_name</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">last_name</span> <span class="o">=</span> <span class="s">'Bar</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span>
<span class="k">if</span> <span class="n">instrument</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">instrument</span> <span class="o">=</span> <span class="s">'Blowfish</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span>
<span class="n">musician</span> <span class="o">=</span> <span class="n">Musician</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">first_name</span><span class="o">=</span><span class="n">first_name</span><span class="p">,</span>
<span class="n">last_name</span><span class="o">=</span><span class="n">last_name</span><span class="p">,</span>
<span class="n">instrument</span><span class="o">=</span><span class="n">instrument</span>
<span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span><span class="mi">1</span>
<span class="k">return</span> <span class="n">musician</span>
<span class="k">class</span> <span class="nc">AlbumFactory</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">=</span> <span class="mi">0</span>
<span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">musician</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">release_date</span><span class="o">=</span><span class="bp">None</span><span class="p">,</span> <span class="n">num_stars</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
<span class="k">if</span> <span class="n">name</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">name</span> <span class="o">=</span> <span class="s">'Album</span><span class="si">%s</span><span class="s">'</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">counter</span>
<span class="k">if</span> <span class="n">release_date</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">release_date</span> <span class="o">=</span> <span class="s">'2014-03-03'</span>
<span class="k">if</span> <span class="n">num_stars</span> <span class="ow">is</span> <span class="bp">None</span><span class="p">:</span>
<span class="n">num_stars</span> <span class="o">=</span> <span class="mi">5</span>
<span class="n">album</span> <span class="o">=</span> <span class="n">Album</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span>
<span class="n">musician</span><span class="o">=</span><span class="n">musician</span><span class="p">,</span>
<span class="n">name</span><span class="o">=</span><span class="n">name</span><span class="p">,</span>
<span class="n">release_date</span><span class="o">=</span><span class="n">release_date</span><span class="p">,</span>
<span class="n">num_stars</span><span class="o">=</span><span class="n">num_stars</span>
<span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="n">album</span>
</pre></div>
<p>When creating records in db for testing, we can write in <cite>setUp</cite> something as:</p>
<div class="highlight"><pre><span class="k">def</span> <span class="nf">setUp</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">musician</span> <span class="o">=</span> <span class="n">MusicianFactory</span><span class="p">()</span>
<span class="n">album</span> <span class="o">=</span> <span class="n">AlbumFactory</span><span class="p">()</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">):</span>
<span class="n">album</span><span class="p">(</span><span class="n">musician</span><span class="o">=</span><span class="n">musician</span><span class="p">())</span>
</pre></div>
<p>This approach will bring us a few benefits, such as:</p>
<ul class="simple">
<li>creating objects with default data</li>
<li>creating sequence of objects</li>
<li>focus on tests not on creating records in db</li>
</ul>
<p>At any time, you can override the default data with data specific to your
testing context:</p>
<div class="highlight"><pre><span class="n">musician</span> <span class="o">=</span> <span class="n">MusicianFactory</span><span class="p">()</span>
<span class="n">musician</span> <span class="o">=</span> <span class="n">musician</span><span class="p">(</span><span class="n">first_name</span><span class="o">=</span><span class="s">'x'</span><span class="p">,</span> <span class="n">last_name</span><span class="o">=</span><span class="s">'y'</span><span class="p">)</span>
</pre></div>
<p>After completing this tutorial, we have a good foundation to go and start
testing an API from a functional point of view.</p>
<p>One thing you'll notice using the above examples will be the repeatable code
for each model when creating a factory. We can see a pattern, so let's try to
refactor a little bit the factory and create a more generic one:</p>
<div class="highlight"><pre><span class="k">class</span> <span class="nc">ModelFactory</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">model</span><span class="p">,</span> <span class="o">**</span><span class="n">fields</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_model</span> <span class="o">=</span> <span class="n">model</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_fields</span> <span class="o">=</span> <span class="n">fields</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_counter</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">def</span> <span class="nf">__call__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
<span class="n">fields</span> <span class="o">=</span> <span class="nb">dict</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">_fields</span><span class="p">)</span>
<span class="n">fields</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">kwargs</span><span class="p">)</span>
<span class="n">f</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">fields</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
<span class="k">if</span> <span class="nb">callable</span><span class="p">(</span><span class="n">v</span><span class="p">):</span>
<span class="n">new_v</span> <span class="o">=</span> <span class="n">v</span>
<span class="k">try</span><span class="p">:</span>
<span class="n">new_v</span> <span class="o">=</span> <span class="n">v</span> <span class="o">%</span> <span class="bp">self</span><span class="o">.</span><span class="n">_counter</span>
<span class="k">except</span> <span class="ne">TypeError</span><span class="p">:</span>
<span class="n">new_v</span> <span class="o">=</span> <span class="n">v</span>
<span class="n">f</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="n">new_v</span>
<span class="bp">self</span><span class="o">.</span><span class="n">_counter</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_model</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">create</span><span class="p">(</span><span class="o">**</span><span class="n">f</span><span class="p">)</span>
</pre></div>
<p>Now it't more simpler and easier to create factories</p>
<div class="highlight"><pre><span class="n">musician</span> <span class="o">=</span> <span class="n">ModelFactory</span><span class="p">(</span>
<span class="n">Musician</span><span class="p">,</span> <span class="n">first_name</span><span class="o">=</span><span class="s">'Foo'</span><span class="p">,</span> <span class="n">last_name</span><span class="o">=</span><span class="s">'Bar'</span><span class="p">,</span> <span class="n">instrument</span><span class="o">=</span><span class="s">'blowfish'</span>
<span class="p">)</span>
<span class="n">album</span> <span class="o">=</span> <span class="n">ModelFactory</span><span class="p">(</span>
<span class="n">Album</span><span class="p">,</span>
<span class="n">musician</span><span class="o">=</span><span class="n">musician</span><span class="p">(),</span>
<span class="n">name</span><span class="o">=</span><span class="s">'Album</span><span class="si">%s</span><span class="s">'</span><span class="p">,</span>
<span class="n">release_date</span><span class="o">=</span><span class="s">'2014-02-02'</span><span class="p">,</span>
<span class="n">num_stars</span><span class="o">=</span><span class="s">'</span><span class="si">%s</span><span class="s">'</span>
<span class="p">)</span>
<span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">):</span>
<span class="n">album</span><span class="p">()</span>
</pre></div>
<p>This will create a musician with 3 albums</p>
<div class="highlight"><pre><span class="p">[{</span><span class="s">u'albums'</span><span class="p">:</span> <span class="p">[{</span><span class="s">u'id'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">u'name'</span><span class="p">:</span> <span class="s">u'Album1'</span><span class="p">,</span>
<span class="s">u'num_stars'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">u'release_date'</span><span class="p">:</span> <span class="s">u'2014-02-02'</span><span class="p">},</span>
<span class="p">{</span><span class="s">u'id'</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span>
<span class="s">u'name'</span><span class="p">:</span> <span class="s">u'Album2'</span><span class="p">,</span>
<span class="s">u'num_stars'</span><span class="p">:</span> <span class="mi">2</span><span class="p">,</span>
<span class="s">u'release_date'</span><span class="p">:</span> <span class="s">u'2014-02-02'</span><span class="p">},</span>
<span class="p">{</span><span class="s">u'id'</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span>
<span class="s">u'name'</span><span class="p">:</span> <span class="s">u'Album3'</span><span class="p">,</span>
<span class="s">u'num_stars'</span><span class="p">:</span> <span class="mi">3</span><span class="p">,</span>
<span class="s">u'release_date'</span><span class="p">:</span> <span class="s">u'2014-02-02'</span><span class="p">}],</span>
<span class="s">u'first_name'</span><span class="p">:</span> <span class="s">u'Foo'</span><span class="p">,</span>
<span class="s">u'id'</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
<span class="s">u'instrument'</span><span class="p">:</span> <span class="s">u'blowfish'</span><span class="p">,</span>
<span class="s">u'last_name'</span><span class="p">:</span> <span class="s">u'Bar'</span><span class="p">}]</span>
</pre></div>
Circular foreign key with factory_boy2014-02-09T00:00:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2014-02-09:circular-foreign-key-factory-boy/<p>Given the following state:</p>
<div class="highlight"><pre><span class="k">class</span> <span class="nc">Album</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">thumb</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s">'Image'</span><span class="p">,</span> <span class="n">related_name</span><span class="o">=</span><span class="s">'image'</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">Image</span><span class="p">(</span><span class="n">models</span><span class="o">.</span><span class="n">Model</span><span class="p">):</span>
<span class="n">album</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">ForeignKey</span><span class="p">(</span><span class="s">'Album'</span><span class="p">,</span> <span class="n">null</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">blank</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</pre></div>
<p>with factory_boy:</p>
<div class="highlight"><pre><span class="k">class</span> <span class="nc">AlbumFactory</span><span class="p">(</span><span class="n">factory</span><span class="o">.</span><span class="n">DjangoModelFactory</span><span class="p">):</span>
<span class="n">FACTORY_FOR</span> <span class="o">=</span> <span class="s">'app.Album'</span>
<span class="n">thumb</span> <span class="o">=</span> <span class="n">factory</span><span class="o">.</span><span class="n">SubFactory</span><span class="p">(</span>
<span class="s">'path.to.factories.ImageFactory'</span>
<span class="p">)</span>
<span class="k">class</span> <span class="nc">ImageFactory</span><span class="p">(</span><span class="n">factory</span><span class="o">.</span><span class="n">DjangoModelFactory</span><span class="p">):</span>
<span class="n">FACTORY_FOR</span> <span class="o">=</span> <span class="s">'app.Image'</span>
<span class="n">album</span> <span class="o">=</span> <span class="n">factory</span><span class="o">.</span><span class="n">RelatedFactory</span><span class="p">(</span><span class="n">AlbumFactory</span><span class="p">,</span> <span class="s">'thumb'</span><span class="p">)</span>
</pre></div>
Autospeccing your mocks2014-02-03T00:00:00+02:00Dan Claudiu Poptag:blog.danclaudiupop.com,2014-02-03:auto-speccing/<p>Auto-speccing can be done through the <cite>autospec</cite> argument to patch, or the
<cite>create_autospec</cite> function. Auto-speccing creates mock objects that have the
same attributes and methods as the objects they are replacing, and any
functions and methods (including constructors) have the same call signature as
the real object.</p>
<p>Given that, if your mock is used in illegal ways, for e.g a mock method is
called with incorrect number of arguments, an exception will be thrown. When
refactoring is happening and your tests still serve as living documentation,
tests that are using mocks without autospeccing flag will continue to pass.</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">mock</span> <span class="kn">import</span> <span class="n">create_autospec</span>
<span class="kn">import</span> <span class="nn">unittest</span>
<span class="k">def</span> <span class="nf">foo</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">class</span> <span class="nc">FooTestCase</span><span class="p">(</span><span class="n">unittest</span><span class="o">.</span><span class="n">TestCase</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">test_foo_spec</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">mock</span> <span class="o">=</span> <span class="n">create_autospec</span><span class="p">(</span><span class="n">foo</span><span class="p">)</span>
<span class="n">mock</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">mock</span><span class="o">.</span><span class="n">assert_called_with</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">mock</span><span class="o">.</span><span class="n">return_value</span> <span class="o">=</span> <span class="mi">10</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertEqual</span><span class="p">(</span><span class="n">mock</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="mi">10</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">assertRaises</span><span class="p">(</span><span class="ne">TypeError</span><span class="p">,</span> <span class="n">mock</span><span class="p">,</span> <span class="s">'only_one_arg'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">'__main__'</span><span class="p">:</span>
<span class="n">unittest</span><span class="o">.</span><span class="n">main</span><span class="p">()</span>
</pre></div>
Headless selenium tests in django with xvfbwrapper2013-11-29T00:00:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2013-11-29:headless-selenium-tests-in-django-with-xvfbwrapper/<p>Install dependencies:</p>
<ul class="simple">
<li><tt class="docutils literal">sudo <span class="pre">apt-get</span> install xvfb</tt></li>
<li><tt class="docutils literal">pip install xvfbwrapper</tt></li>
</ul>
<p>This code uses selenium and xvfbwrapper to run tests with Firefox inside a
headless display.</p>
<div class="highlight"><pre><span class="kn">from</span> <span class="nn">django.test</span> <span class="kn">import</span> <span class="n">LiveServerTestCase</span>
<span class="kn">from</span> <span class="nn">selenium.webdriver.firefox.webdriver</span> <span class="kn">import</span> <span class="n">WebDriver</span>
<span class="kn">from</span> <span class="nn">xvfbwrapper</span> <span class="kn">import</span> <span class="n">Xvfb</span>
<span class="k">class</span> <span class="nc">SeleniumTestCase</span><span class="p">(</span><span class="n">LiveServerTestCase</span><span class="p">):</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">setUpClass</span><span class="p">(</span><span class="n">cls</span><span class="p">):</span>
<span class="n">cls</span><span class="o">.</span><span class="n">xvfb</span> <span class="o">=</span> <span class="n">Xvfb</span><span class="p">(</span><span class="n">width</span><span class="o">=</span><span class="mi">1280</span><span class="p">,</span> <span class="n">height</span><span class="o">=</span><span class="mi">720</span><span class="p">)</span>
<span class="n">cls</span><span class="o">.</span><span class="n">xvfb</span><span class="o">.</span><span class="n">start</span><span class="p">()</span>
<span class="n">cls</span><span class="o">.</span><span class="n">wd</span> <span class="o">=</span> <span class="n">WebDriver</span><span class="p">()</span>
<span class="nb">super</span><span class="p">(</span><span class="n">SeleniumTestCase</span><span class="p">,</span> <span class="n">cls</span><span class="p">)</span><span class="o">.</span><span class="n">setUpClass</span><span class="p">()</span>
<span class="nd">@classmethod</span>
<span class="k">def</span> <span class="nf">tearDownClass</span><span class="p">(</span><span class="n">cls</span><span class="p">):</span>
<span class="n">cls</span><span class="o">.</span><span class="n">wd</span><span class="o">.</span><span class="n">quit</span><span class="p">()</span>
<span class="nb">super</span><span class="p">(</span><span class="n">SeleniumTestCase</span><span class="p">,</span> <span class="n">cls</span><span class="p">)</span><span class="o">.</span><span class="n">tearDownClass</span><span class="p">()</span>
<span class="n">cls</span><span class="o">.</span><span class="n">xvfb</span><span class="o">.</span><span class="n">stop</span><span class="p">()</span>
</pre></div>
10 pragmatic guidelines to maximize test usefulness2013-11-05T00:00:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2013-11-05:10-pragmatic-guidelines-to-maximize-test-usefulness/<iframe width="420" height="315" src="//www.youtube.com/embed/91-LiEb3sPE" frameborder="0" allowfullscreen></iframe><div class="line-block">
<div class="line"><br /></div>
</div>
<div class="section" id="abstract">
<h2>ABSTRACT</h2>
<p><strong>WHAT MAKES GOOD TESTS?</strong></p>
<ul class="simple">
<li>Tests should be as simple as possible.</li>
<li>Tests should run as quickly as possible.</li>
<li>Tests should avoid coupling with other tests.</li>
<li>Tests should communicate intent.</li>
</ul>
<p><strong>CANONICAL FORM OF A TEST</strong></p>
<ul class="simple">
<li>Setup pre-conditions.</li>
<li>Perform operation under test.</li>
<li>Make assertions.</li>
<li>Benefits of following canonical form.</li>
</ul>
<p><strong>TESTING TOOLS</strong></p>
<ul class="simple">
<li>Factories (instead of fixtures).</li>
<li>Coverage.py</li>
<li>django-nose + multiprocess</li>
</ul>
<p><strong>GUIDELINES</strong></p>
<ul class="simple">
<li>Tests should be as simple as possible.</li>
<li>Each test method tests one thing, and one thing only.</li>
<li>Only set up the minimum needed pre-conditions for your test.</li>
<li>Create your pre-conditions explicitly - don't use shared helper methods outside your module.</li>
<li>Name your TestCase methods to indicate what they actually test.</li>
<li>Use factories, not fixtures.</li>
<li>Use django.tests.TestCase instead of unittest2.TestCase</li>
<li>Create mixins, not shared TestCases.</li>
<li>Segment your tests.</li>
<li>Don't use setupClass() or tearDownClass()</li>
</ul>
</div>
Stop treating testers as failed developers2013-10-16T11:30:00+03:00danclaudiupoptag:blog.danclaudiupop.com,2013-10-16:stop-treating-testers-as-failed-developers/<p><em>"Stop treating testers as failed developers when developers are failed
testers"</em></p>
<p>Thanks to <a class="reference external" href="https://github.com/limpangel">Angel Ramboi</a> who took the picture
at PyCon Ireland 2013. This is <a class="reference external" href="https://github.com/AutomatedTester">David Burns</a>' keynote.</p>
<a href="http://www.flickr.com/photos/105549603@N05/10305606266/"><img src="http://farm8.staticflickr.com/7411/10305606266_3138beee0d_b.jpg" alt="IMG_20131013_130435"></a>Command line JSON parser via jq2013-10-14T22:30:00+03:00danclaudiupoptag:blog.danclaudiupop.com,2013-10-14:command-line-json-parser-via-jq/<p>A command-line JSON parser can be handy when you test or debug JSON web
services. Unfortunately inspecting JSON responses via command line are hard to
read and not easy to manipulate with traditional Unix utilities.</p>
<p>Today I stumbled on <a class="reference external" href="http://stedolan.github.io/jq/">jq</a>, via <a class="reference external" href="https://github.com/limpangel">Angel Ramboi</a>. <tt class="docutils literal">jq</tt> is a lightweight and flexible
command-line JSON processor.</p>
<p>Download the desired binary and then <tt class="docutils literal">chmod +x ./jq</tt>.</p>
<p>I've prepared a sample json so we can see how easily it is to inspect and
manipulate JSON strings.</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span>
<span class="p">{</span>
<span class="s">"first"</span><span class="p">:</span> <span class="s">"John"</span><span class="p">,</span>
<span class="s">"last"</span><span class="p">:</span> <span class="s">"Doe"</span><span class="p">,</span>
<span class="s">"age"</span><span class="p">:</span> <span class="mi">27</span><span class="p">,</span>
<span class="s">"sex"</span><span class="p">:</span> <span class="s">"M"</span><span class="p">,</span>
<span class="s">"registered"</span><span class="p">:</span> <span class="n">true</span><span class="p">,</span>
<span class="s">"interests"</span><span class="p">:</span> <span class="p">[</span> <span class="s">"Reading"</span><span class="p">,</span> <span class="s">"Mountain Biking"</span><span class="p">,</span> <span class="s">"Hacking"</span> <span class="p">],</span>
<span class="s">"favorites"</span><span class="p">:</span> <span class="p">{</span>
<span class="s">"color"</span><span class="p">:</span> <span class="s">"Blue"</span><span class="p">,</span>
<span class="s">"sport"</span><span class="p">:</span> <span class="s">"Soccer"</span><span class="p">,</span>
<span class="s">"food"</span><span class="p">:</span> <span class="s">"Spaghetti"</span>
<span class="p">},</span>
<span class="s">"skills"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">"category"</span><span class="p">:</span> <span class="s">"Python"</span><span class="p">,</span>
<span class="s">"tests"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span> <span class="s">"name"</span><span class="p">:</span> <span class="s">"One"</span><span class="p">,</span> <span class="s">"score"</span><span class="p">:</span> <span class="mi">90</span> <span class="p">},</span>
<span class="p">{</span> <span class="s">"name"</span><span class="p">:</span> <span class="s">"Two"</span><span class="p">,</span> <span class="s">"score"</span><span class="p">:</span> <span class="mi">96</span> <span class="p">}</span>
<span class="p">]</span>
<span class="p">},</span>
<span class="p">{</span>
<span class="s">"category"</span><span class="p">:</span> <span class="s">"GO"</span><span class="p">,</span>
<span class="s">"tests"</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span> <span class="s">"name"</span><span class="p">:</span> <span class="s">"One"</span><span class="p">,</span> <span class="s">"score"</span><span class="p">:</span> <span class="mi">32</span> <span class="p">},</span>
<span class="p">{</span> <span class="s">"name"</span><span class="p">:</span> <span class="s">"Two"</span><span class="p">,</span> <span class="s">"score"</span><span class="p">:</span> <span class="mi">84</span> <span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</pre></div>
<p>Simple filtering:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span> <span class="o">|</span> <span class="n">jq</span> <span class="s">'.skills[].category'</span>
<span class="n">Python</span>
<span class="n">Go</span>
</pre></div>
<p>Filtering with index:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span> <span class="o">|</span> <span class="o">./</span><span class="n">jq</span> <span class="s">'.["skills"][0]["tests"][1] | .name, .score'</span>
<span class="s">"Two"</span>
<span class="mi">96</span>
</pre></div>
<p>...or a different flavour:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span> <span class="o">|</span> <span class="o">./</span><span class="n">jq</span> <span class="s">'.skills[0].tests[1] | .name, .score'</span>
<span class="s">"Two"</span>
<span class="mi">96</span>
</pre></div>
<p>Built-in operators:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span> <span class="o">|</span> <span class="o">./</span><span class="n">jq</span> <span class="s">'.interests | length'</span>
<span class="mi">3</span>
</pre></div>
<p>Manipulate JSON string:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">cat</span> <span class="n">sample</span><span class="o">.</span><span class="n">json</span> <span class="o">|</span> <span class="o">./</span><span class="n">jq</span> <span class="s">'if .registered == true then .skills[].tests[].score = 1000 else null end'</span> <span class="o">></span> <span class="n">new_sample</span><span class="o">.</span><span class="n">json</span>
</pre></div>
<p>Head on and read the <a class="reference external" href="http://stedolan.github.io/jq/manual/">jq manual</a> if you
want to include it in your tester toolbox.</p>
Using tail to explore logs2013-10-03T12:30:00+03:00danclaudiupoptag:blog.danclaudiupop.com,2013-10-03:using-tail-to-explore-logs/<p>When I am testing, I always keep an eye on the logs because they are a valuable
resource for finding problems on the system. They can act as an early warning
system. The <tt class="docutils literal">tail</tt> commnand alongside with <tt class="docutils literal">view</tt> or <tt class="docutils literal">vim</tt> are great ways
to explore big log entries.</p>
<p>Some simple examples:</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">tail</span> <span class="o">-</span><span class="n">f</span> <span class="n">web_app</span><span class="o">.</span><span class="n">log</span> <span class="n">import</span><span class="o">.</span><span class="n">log</span>
<span class="o">>></span> <span class="n">tail</span> <span class="o">-</span><span class="n">f</span> <span class="o">*.</span><span class="n">log</span>
</pre></div>
<div class="highlight"><pre><span class="o">>></span> <span class="n">tail</span> <span class="o">-</span><span class="n">f</span> <span class="n">web_app</span><span class="o">.</span><span class="n">log</span> <span class="n">import</span><span class="o">.</span><span class="n">log</span> <span class="o">|</span> <span class="n">grep</span> <span class="n">ERROR</span>
</pre></div>
<p>... or get and read log from remote host</p>
<div class="highlight"><pre><span class="o">>></span> <span class="n">ssh</span> <span class="n">remoteUser</span><span class="nd">@remoteHost</span> <span class="s">"tail -f /var/log/app/app.log"</span> <span class="o">|</span> <span class="n">tee</span> <span class="n">local</span><span class="o">.</span><span class="n">log</span>
</pre></div>
<p><em>less is more</em></p>
<p>More convenient way to monitor logs is by using less. <tt class="docutils literal">less + F app.log</tt>
allows you to switch between watch mode and navigation mode with <tt class="docutils literal">ctrl+c</tt> and
back to watching mode with <tt class="docutils literal">F</tt>. Very helpul when you need to search, find
occurrences, create marks etc.</p>
After Romanian Testing Conference 20132013-05-24T00:00:00+03:00danclaudiupoptag:blog.danclaudiupop.com,2013-05-24:rtc/<p>It’s been awhile since myself and my colleague Dan Claudiu Pop have had our
presentation at RTC, but given that this experience has had an impact on our
testing perspective, we decided to write a few lines on how it was and what it
meant to us.</p>
<img alt="Romanian Testing Conference 2013" src="/images/rtc_2013.png" />
<p>We were pleasantly surprised to see that many colleagues share our opinions on
software testing and this alone was enough to get us through the emotions we
had that day, simply because we knew that we were speaking the same language.
More and more software testers believe in the unity of the team, without the
need of separating the tester (improperly called QA by many still) and
developer roles.</p>
<p>We decided to talk about “Our experience with BDD in Python”, and throughout
the presentation, we did exactly that: we talked about BDD as we understood and
practiced it. We wanted to avoid “best practices” and “how to” terminology,
because we didn’t want to teach others how to apply this methodology. We simply
wanted to share our knowledge on the item, how we applied it and what are the
challenges we are facing with.</p>
<p>Shortly after the presentation has begun, we understood that this was the
correct approach. People were drawn to what we had to say and we felt that the
issues we were exposing were issues they also had encountered.</p>
<p>I don’t know if we managed to answer all of their questions, but as long as we
got the audience thinking about our subject and comparing the items we
presented with the challenges they are facing day to day, then the presentation
was a success.</p>
<p>There’s no golden formula which, when applied, it magically solves all issues
on a project. But as long as everybody has understood our message -
communication and collaboration is everything - then we definitely can say we
are on the right track.</p>
<p>You can see the presentation here - <a class="reference external" href="http://testpy.github.io/RTC-presentation/">http://testpy.github.io/RTC-presentation/</a></p>
Functional tests with django-webtest2013-04-25T00:00:00+03:00Dan Claudiu Poptag:blog.danclaudiupop.com,2013-04-25:functional-tests-with-django-webtest/<p>I’ve been watching both presentations that Carl Meyer held at Pycon 2012/13 and
I highly recommend them if you want a deep dive into writing tests with django.
They outline some very good principles for writing effective and maintainable
tests. They also highlight a suite of test utilities and frameworks which help
you in writing better tests. Among the others, Webtest caught my attention via
<a class="reference external" href="https://github.com/kmike/django-webtest">django-webtest</a> for writing
integration/functional tests.</p>
<div class="highlight"><pre><span class="k">def</span> <span class="nf">testLoginProcess</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">login</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">app</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">reverse</span><span class="p">(</span><span class="s">'auth_login'</span><span class="p">))</span>
<span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="p">[</span><span class="s">'username'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'danu'</span>
<span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="p">[</span><span class="s">'password'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'test123'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="s">'Log in'</span><span class="p">)</span><span class="o">.</span><span class="n">follow</span><span class="p">()</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="s">'200 OK'</span><span class="p">,</span> <span class="n">response</span><span class="o">.</span><span class="n">status</span><span class="p">)</span>
<span class="n">assert_contains</span><span class="p">(</span><span class="n">response</span><span class="p">,</span> <span class="s">'Welcome danu :]'</span><span class="p">,</span> <span class="n">count</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">status_code</span><span class="o">=</span><span class="mi">200</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">testLoginWithInvalidCredentials</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">login</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">app</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">reverse</span><span class="p">(</span><span class="s">'auth_login'</span><span class="p">))</span>
<span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="p">[</span><span class="s">'username'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'foo'</span>
<span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="p">[</span><span class="s">'password'</span><span class="p">]</span> <span class="o">=</span> <span class="s">'bar'</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">login</span><span class="o">.</span><span class="n">form</span><span class="o">.</span><span class="n">submit</span><span class="p">(</span><span class="s">'Log in'</span><span class="p">)</span>
<span class="n">assert_contains</span><span class="p">(</span>
<span class="n">response</span><span class="p">,</span>
<span class="s">'Please enter a correct username and password. '</span>
<span class="s">'Note that both fields are case-sensitive.'</span><span class="p">,</span>
<span class="n">count</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span>
<span class="n">status_code</span><span class="o">=</span><span class="mi">200</span>
<span class="p">)</span>
</pre></div>
<p><strong>Why you should choose django-webtest instead of django client ?</strong> Well,
first of all, it can better capture the user experience mainly because you can
submit forms and follow links. You are not only testing the views but also the
template html. Secondly, it will allow you to write more simple and readable
tests, an important fact when it comes to integration or functional tests.</p>
<p>It interacts with django via the WSGI interface so ajax, js will not be tested.
For that purpose you normally use selenium.</p>
<p>I’ve set up a project on <a class="reference external" href="https://github.com/danclaudiupop/django-lab-tests">github</a> to illustrate some
automated tests with django-webtest. It uses nose test suite runner since i
don’t want to to import all my tests into tests/__init__.py and for rest of the
goodness that nose offers.</p>
Lettuce and django test client2013-02-01T00:00:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2013-02-01:lettuce-and-django-test-client/<p>When running a functional test, you fire up “browser” and do the "same" actions
as a real user (or API client). There are different “browsers” for testing your
applications, some of them are real, like selenium and some of them are less
real, like django test client. Depending on the context, each of them has its
pros and cons.</p>
<ul class="simple">
<li>django test client is very fast, since you don’t need a browser engine, real
http listening and so on, but JavaScript and/or Ajax views are not tested</li>
<li>selenium is a browser-driving library, opens a real browser and tests
rendered HTML alongside with behavior of Web pages. Write selenium tests for
Ajax, other JS/server interactions.</li>
</ul>
<p>A complete test suite should contain both test types.</p>
<p>Write functional/integration/system (has more names than it needs) tests for
views. Unit testing the view is hard because views have many dependencies
(templates, db, middleware, url routing etc).</p>
<p>You should definitely have django functional tests but chances are you should
have fewer than you have now. See the <a class="reference external" href="https://github.com/kmike/django-webtest">software testing pyramid</a> by Alister Scott which provides
solid approach to automated testing and shows the mix of testing a team should
aim for.</p>
<p><em>Functional tests:</em></p>
<ul class="simple">
<li>test that the whole integrated system works, catch regressions</li>
<li>they tend to be slow</li>
<li>will catch bugs that unit tests will not, but it's harder to debug</li>
<li>write fewer (more unit tests)</li>
</ul>
<p><em>So what can you test with django-test client ?</em></p>
<ul class="simple">
<li>the correct view is executed for a given url</li>
<li>simulate post, get, head, put etc. requests</li>
<li>the returned content has the expected values (you can use beautiful soup,
lxml or html5lib for parsing the content)</li>
</ul>
<p><strong>Example</strong></p>
<div class="highlight"><pre><span class="n">Feature</span><span class="p">:</span> <span class="n">Register</span>
<span class="n">In</span> <span class="n">order</span> <span class="n">to</span> <span class="n">get</span> <span class="n">access</span> <span class="n">to</span> <span class="n">app</span>
<span class="n">A</span> <span class="n">user</span> <span class="n">should</span> <span class="n">be</span> <span class="n">able</span> <span class="n">to</span> <span class="n">register</span>
<span class="n">Scenario</span><span class="p">:</span> <span class="n">User</span> <span class="n">registers</span>
<span class="n">Given</span> <span class="n">I</span> <span class="n">go</span> <span class="n">to</span> <span class="n">the</span> <span class="n">registration</span> <span class="n">page</span>
<span class="n">When</span> <span class="n">I</span> <span class="n">fill</span> <span class="n">register</span> <span class="n">form</span> <span class="k">with</span><span class="p">:</span>
<span class="o">|</span> <span class="n">username</span> <span class="o">|</span> <span class="n">email</span> <span class="o">|</span> <span class="n">password1</span> <span class="o">|</span> <span class="n">password2</span> <span class="o">|</span>
<span class="o">|</span> <span class="n">danul</span> <span class="o">|</span> <span class="n">dan</span><span class="nd">@dan.com</span> <span class="o">|</span> <span class="n">test123</span> <span class="o">|</span> <span class="n">test123</span> <span class="o">|</span>
<span class="n">And</span> <span class="n">I</span> <span class="n">submit</span> <span class="n">the</span> <span class="n">data</span>
<span class="n">Then</span> <span class="n">I</span> <span class="n">should</span> <span class="n">see</span> <span class="s">"Check your email"</span>
<span class="n">And</span> <span class="n">I</span> <span class="n">should</span> <span class="n">receive</span> <span class="n">an</span> <span class="n">email</span> <span class="n">at</span> <span class="s">"dan@dan.com"</span> <span class="k">with</span> <span class="n">the</span> <span class="n">subject</span> <span class="s">"Activate your djangoproject.com account - you have 7 days!"</span>
<span class="n">And</span> <span class="n">I</span> <span class="n">activate</span> <span class="n">the</span> <span class="n">account</span>
<span class="n">Then</span> <span class="n">I</span> <span class="n">should</span> <span class="n">see</span> <span class="s">"Congratulations!"</span>
</pre></div>
<div class="highlight"><pre><span class="nd">@step</span><span class="p">(</span><span class="s">u'I go to the register page'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">i_go_to_the_register_page</span><span class="p">(</span><span class="n">step</span><span class="p">):</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">world</span><span class="o">.</span><span class="n">browser</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">reverse</span><span class="p">(</span><span class="s">'registration_register'</span><span class="p">))</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="n">world</span><span class="o">.</span><span class="n">html</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">response</span><span class="o">.</span><span class="n">content</span><span class="p">)</span>
<span class="nd">@step</span><span class="p">(</span><span class="s">u'When I fill register form with:'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">when_i_fill_in_user_data_with</span><span class="p">(</span><span class="n">step</span><span class="p">):</span>
<span class="k">for</span> <span class="n">data</span> <span class="ow">in</span> <span class="n">step</span><span class="o">.</span><span class="n">hashes</span><span class="p">:</span>
<span class="n">world</span><span class="o">.</span><span class="n">data</span> <span class="o">=</span> <span class="n">data</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">world</span><span class="o">.</span><span class="n">html</span><span class="o">.</span><span class="n">select</span><span class="p">(</span><span class="s">'form'</span><span class="p">)),</span> <span class="mi">1</span><span class="p">)</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">world</span><span class="o">.</span><span class="n">html</span><span class="o">.</span><span class="n">find_all</span><span class="p">(</span><span class="s">'input'</span><span class="p">,</span> <span class="s">'required'</span><span class="p">)),</span> <span class="mi">4</span><span class="p">)</span>
<span class="nd">@step</span><span class="p">(</span><span class="s">u'And I submit the data'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">and_i_submit_the_data</span><span class="p">(</span><span class="n">step</span><span class="p">):</span>
<span class="n">world</span><span class="o">.</span><span class="n">response</span> <span class="o">=</span> <span class="n">world</span><span class="o">.</span><span class="n">browser</span><span class="o">.</span><span class="n">post</span><span class="p">(</span>
<span class="n">reverse</span><span class="p">(</span><span class="s">'registration_register'</span><span class="p">),</span>
<span class="n">world</span><span class="o">.</span><span class="n">data</span><span class="p">,</span>
<span class="n">follow</span><span class="o">=</span><span class="bp">True</span>
<span class="p">)</span>
<span class="n">assert_equals</span><span class="p">(</span>
<span class="n">User</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="n">filter</span><span class="p">(</span><span class="n">username</span><span class="o">=</span><span class="n">world</span><span class="o">.</span><span class="n">data</span><span class="p">[</span><span class="s">'username'</span><span class="p">])</span><span class="o">.</span><span class="n">exists</span><span class="p">(),</span> <span class="bp">True</span>
<span class="p">)</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">world</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
<span class="nd">@step</span><span class="p">(</span><span class="s">u'I should see "(.*)"'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">i_should_see</span><span class="p">(</span><span class="n">step</span><span class="p">,</span> <span class="n">expected_response</span><span class="p">):</span>
<span class="n">html</span> <span class="o">=</span> <span class="n">BeautifulSoup</span><span class="p">(</span><span class="n">world</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">content</span><span class="p">)</span>
<span class="n">expected_text</span> <span class="o">=</span> <span class="n">html</span><span class="o">.</span><span class="n">find</span><span class="p">(</span><span class="s">'h1'</span><span class="p">)</span><span class="o">.</span><span class="n">get_text</span><span class="p">()</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">expected_text</span><span class="p">,</span> <span class="n">expected_response</span><span class="p">)</span>
<span class="nd">@step</span><span class="p">(</span><span class="s">u'And I should receive an email at "([^"]*)" with the subject "([^"]*)"'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">i_should_receive_email_with_subject</span><span class="p">(</span><span class="n">step</span><span class="p">,</span> <span class="n">address</span><span class="p">,</span> <span class="n">subject</span><span class="p">):</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">mail</span><span class="o">.</span><span class="n">outbox</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">to</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">address</span><span class="p">)</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">mail</span><span class="o">.</span><span class="n">outbox</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">subject</span><span class="p">,</span> <span class="n">subject</span><span class="p">)</span>
<span class="nd">@step</span><span class="p">(</span><span class="s">u'And I activate the account'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">and_i_activate_the_account</span><span class="p">(</span><span class="n">step</span><span class="p">):</span>
<span class="n">activation_url</span> <span class="o">=</span> <span class="n">re</span><span class="o">.</span><span class="n">findall</span><span class="p">(</span>
<span class="s">r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+'</span><span class="p">,</span>
<span class="n">mail</span><span class="o">.</span><span class="n">outbox</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">body</span>
<span class="p">)</span>
<span class="n">world</span><span class="o">.</span><span class="n">response</span> <span class="o">=</span> <span class="n">world</span><span class="o">.</span><span class="n">browser</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">activation_url</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">follow</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">assert_equals</span><span class="p">(</span><span class="n">world</span><span class="o">.</span><span class="n">response</span><span class="o">.</span><span class="n">status_code</span><span class="p">,</span> <span class="mi">200</span><span class="p">)</span>
</pre></div>
<p><strong>What's next ?</strong></p>
<p>Well ... WebTest :)</p>
<iframe width="420" height="315" src="//www.youtube.com/embed/ickNQcNXiS4" frameborder="0" allowfullscreen></iframe><div class="line-block">
<div class="line"><br /></div>
</div>
<p>Be a good person and write functional tests. Functional testing is something
that every app needs, no testing strategy is complete without high-level tests
to ensure the entire programming system works together.</p>
Book review: Explore it! by Elisabeth Hendrickson2013-01-07T10:20:00+02:00danclaudiupoptag:blog.danclaudiupop.com,2013-01-07:explore-it/<p>There are many ways to test a software product. There are many methodologies
that could help in testing that software product, so that the final release
contains as less bugs as possible. We are not at the point where we could say
that we have found the solution for bug free applications. But we are at the
point where we know that in order to avoid buggy, hard to use, no precise scope
defined applications, we need an approach that will ensure a higher quality of
the product. One of these ways could very well be Exploratory Testing, done
right, done different. Exploratory Testing is important. Exploratory Testing
can be learnt.</p>
<p>In my opinion, these are some of the messages that can be heard when
reading Explore It!, by Elisabeth Hendrickson.</p>
<p>As Cem Kaner himself states, Exploratory Testing is a style of software
testing that emphasizes the personal freedom and responsibility of the
individual tester to continually optimize the quality of his/her work by
treating test-related learning, test design, test execution, and test result
interpretation as mutually supportive activities that run in parallel
throughout the project. That’s a pretty accurate definition. But what does
this mean exactly? How could I learn to be a more creative individual, when all
around me and my work requires 100% analytical thinking? Do I need to be more
creative, or to have a better sense of responsibility in order to execute
exploratory tests that actually find defects and weaknesses?</p>
<p>I’ve started looking for some of the answers I need through “Exploratory
Testing”. I have to say that I found the book incredibly easy to read and
captivating. The real-world examples were in the right places, and I could
better visualize the applicability of a concept presented in a specific
section. However, at some point, I felt that the examples became too detailed
and a more high-level explanation should have sufficed (but this is maybe a
singular opinion, I guess it depends on our own perception on exploratory
testing). The personas concept is something I have never thought of and it’s
definitely something I would like to try with my team someday. The idea of role
playing of well-defined characters, in order to cover more classes of
vulnerabilities is one of the many out-of-the-box examples and concepts that
are defined in this book.</p>
<p>All in all, I sincerely recommend this book for any tester who would like
to better themselves. It is suitable not only for functional testers, but it
also has a few interesting approaches for the non-functional parts of the
system as well. It has great advices on how to perform a good interview and how
to do well on an interview, when faced with testing unfamiliar applications.</p>
<p>What I realized is that I found a few answers, but I have also discovered
new questions and for me, this is more important. It means that the book got me
thinking and questioning - how could I become a better explorer?</p>