slides

z, ?	toggle help (this)
space, →	next slide
shift-space, ←	previous slide
d	toggle debug mode
## <ret>	go to slide #
c, t	table of contents (vi)
f	toggle footer
r	reload slides
n	toggle notes
p	run preshow

loading presentation...

Testing and Django

Carl J Meyer

@carljm

carl@oddbird.net

github.com/carljm/django-testing-slides

github.com/carljm/django-testing-slides/code

Let's start a project.

$ django-admin.py startproject testing .

$ sed -i -e 's/backends\./backends.sqlite3/;' testing/settings.py

$ ./manage.py test
[snip]
----------------------------------------------------------------------
Ran 412 tests in 14.235s

FAILED (errors=2, skipped=1)
Destroying test database for alias 'default'...

(Oops, guess we're not quite ready to release 1.4.) But 412 tests? 14 extra seconds?

I'll never get those 14 seconds back.

Does anyone still need convincing that fast test suites matter?

Not all apps are created equal.

Non-isolated tests break, isolated tests are pointless to run. Integration tests should be written by the integrator.

No problem.

`./manage.py test just my apps please`

Easy to solve with a shell script. But there's more...

`tests/init.py`

from .test_forms import QuoteFormTest
from .test_models import (
    QuoteTest, SourceTest)
from .test_views import (
    AddQuoteTest, EditQuoteTest,
    ListQuotesTest
    )

Django made me do it. (Or worse yet, "import *".)

Django's test discovery

Wastes my time with tests I don't care about.
Requires app tests to be in a single module (resulting in boilerplate imports).
Forces intermingling of tests and non-test code.

But there's good news...

It's easy to change.

unittest2 discovery
TEST_RUNNER setting

The hipsters like nose or py.test, but unittest2 gets the job done.

class DiscoveryRunner(DjangoTestSuiteRunner):
    """A test suite runner using unittest2 discovery."""
    def build_suite(self, test_labels, extra_tests=None,
                    **kwargs):
        suite = None
        discovery_root = settings.TEST_DISCOVERY_ROOT

        if test_labels:
            suite = defaultTestLoader.loadTestsFromNames(
                test_labels)

        if suite is None:
            suite = defaultTestLoader.discover(
                discovery_root,
                top_level_dir=settings.BASE_PATH,
                )

        if extra_tests:
            for test in extra_tests:
                suite.addTest(test)

        return reorder_suite(suite, (TestCase,))

(Better version in the code online with the slides.)

`settings.py`

import os.path

BASE_PATH = os.path.dirname(os.path.dirname(__file__))

TEST_DISCOVERY_ROOT = os.path.join(BASE_PATH, "tests")

TEST_RUNNER = "tests.runner.DiscoveryRunner"

\o/

Discovers tests wherever you want them.
Doesn't run tests from external apps by default.
Flexible specification of specific tests to run: Python dotted path to test module, not Django app label.
./manage.py test tests.quotes.test_views

Maybe in 1.5?

https://code.djangoproject.com/ticket/17365

django-nose is good too, but this could actually go in Django.

Types of test

unit
system / integration / functional

Go see the video of Gary's "Fast test, slow test" talk.

Unit tests

Test one unit of code (a function or method) in something approaching isolation.
Fast, focused (useful failures).
Help you structure your code better.

Integration tests

Test that the whole integrated system works; catch regressions.
Slow.
Less useful failures.
Write fewer.

Summary: both are useful, write more unit tests.

Testing models

The database makes your tests slow.

Try to write tests that don't hit it at all.
Separate db-independent model-layer functionality from db-dependent functionality.
But you'll still have a lot of tests that do.
Mocking the database usually isn't worth it.

class Thing(models.Model):
  def frobnicate(self):
    """Frobnicate and save the thing."""
    # ... do something complicated
    self.save()

def frobnicate_thing(thing):
  # ... do something complicated
  return thing


class Thing(models.Model):
  def frobnicate(self):
    """Frobnicate and save the thing."""
    frobnicate_thing(self)
    self.save()

`django.test.TestCase`

Runs each test within a transaction.
Rolls back the transaction at the end of the test.
Monkeypatches transaction functions to be no-ops.

Django tries to make them fast...

`TransactionTestCase`

Lets you test transactions in your code.
Has to flush every database table after every test.
Makes your tests extra super bonus slow.

This part of the talk is boring because I have no complaints.

{
    "pk": 4,
    "model": "auth.user",
    "fields": {
        "username": "manager",
        "first_name": "",
        "last_name": "",
        "is_active": true,
        "is_superuser": false,
        "is_staff": false,
        "last_login": "2012-02-06 15:06:44",

Do you have these in your tests? BURN THEM!

Just say no.

Hard to maintain and update.
Increase test interdependence.
Slow to load.

Model factories!

def create_profile(**kwargs):
    defaults = {
        "likes_cheese": True,
        "age": 32,
        "address": "3815 Brookside Dr",
    }
    defaults.update(kwargs)
    if "user" not in defaults:
        defaults["user"] = create_user()
    return Profile.objects.create(
        **defaults)

You can write simple factory functions like this (key benefit is prefilling FKs).

Using a factory

def test_can_vote(self):
    """A user age 18+ can vote in the US."""
    profile = create_profile(age=18)
    self.assertTrue(profile.can_vote)

BAD example. This test shouldn't touch the DB. So you want a smarter factory that can also build objects without saving.

Or use `factory_boy`:

class ProfileFactory(factory.Factory):
    FACTORY_FOR = Profile

    likes_cheese = True
    age = 32
    address = "3815 Brookside Dr"
    user = factory.SubFactory(UserFactory)

profile = ProfileFactory.create(
    age=18, user__username="carljm")

Also there's milkman, model_mommy. I don't like random data generation.

Why factories?

Test data local to test code (explicit).
Easy to maintain.
Don't create any data you don't need for that test.
Works great even for large/complex test data sets (helper functions).

Imposing no-DB discipline.

For certain test cases.

from django.utils.unittest import TestCase

import mock

cursor_wrapper = mock.Mock()
cursor_wrapper.side_effect = \
    RuntimeError("No touching the database!")


@mock.patch(
    "django.db.backends.util.CursorWrapper",
    cursor_wrapper)
class NoDBTestCase(TestCase):
    """Will blow up if you database."""

django.utils.unittest.TestCase vs django.test.TestCase. Latter's assertions mostly useful with the DB, but either way works.

Testing views

Unit testing views is hard.

Views have many collaborators / dependencies.
Templates, database, middleware, url routing...
Write less view code!

Views have access to everything, so code in views is easy to write but hard to maintain and debug.

If you unit test views

Use RequestFactory.
Call the view callable directly.
Set up dependencies explicitly (e.g. request.user, request.session).

def test_change_locale(self):
    """POST sets 'locale' key in session."""
    request = RequestFactory().post(
        "/locale/", {"locale": "es-mx"})
    request.session = {}

    change_locale(request)

    self.assertEqual(
        request.session["locale"], "es-mx")

Or don't.

I rarely unit test views.
I write less view code, and cover it via functional tests.

Integration testing views

url = "/case/edit/{0}".format(case.pk)
step = case.steps.get()
response = self.client.post(url, {
    "product": case.product.id,
    "name": case.name,
    "description": case.description,
    "steps-TOTAL_FORMS": 2,
    "steps-INITIAL_FORMS": 1,
    "steps-MAX_NUM_FORMS": 3,
    "steps-0-step": step.step,
    "steps-0-expected": step.expected,
    "steps-1-step": "Click link.",
    "steps-1-expected": "Account active.",
    "status": case.status,
})

or...

WebTest!

url = "/case/edit/{0}".format(case.pk)
form = self.app.get(url).forms["case-form"]
form["steps-1-step"] = "Click link."
form["steps-1-expected"] = "Account active."

response = form.submit()

WebTest parses the form HTML and can submit it like a browser would.

The markup matters.

If it can break, it should be tested.
It can especially break forms.
The output of your view is an HTTP response; the template + context is an implementation detail.

Have to know which markup matters, of course.

WebTest > django.test.Client

System tests are easier and faster to write.
Tests give you more confidence that the view works.
(django-webtest provides integration.)

Django test client is in the "sour spot" - not a unit test, not a full system test. Could gain these features.

self.assertEqual(
   response.json, ["one", "two", "three"])

self.assertEqual(
    resp.html.find("a", title="Login").href,
    "/login/"
    )

Automatically parses JSON or HTML responses (BeautifulSoup or lxml).

In-browser testing

More and more functionality depends on both JS and server. Needs to be tested too.

Is easier than you think.

Especially in Django 1.4.
pip install selenium
LiveServerTestCase

from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver

class MySeleniumTests(LiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        cls.selenium = WebDriver()
        super(MySeleniumTests, cls).setUpClass()

    @classmethod
    def tearDownClass(cls):
        super(MySeleniumTests, cls).tearDownClass()
        cls.selenium.quit()

    def test_login(self):
        self.selenium.get(
            "%s%s" % (self.live_server_url, "/login/"))
        username_input = self.selenium.find_element_by_name(
            "username")
        username_input.send_keys("myuser")
        password_input = self.selenium.find_element_by_name(
            "password")
        password_input.send_keys("secret")
        self.selenium.find_element_by_xpath(
            '//input[@value="Log in"]').click()

LiveServerTestCase runs the development server in a separate thread.

What type of test to write?

Write system tests for your views.
Write Selenium tests for Ajax, other JS/server interactions.
Write unit tests for everything else (not strict).
Test each case (code branch) where it occurs.
One assert/action per test case method.

Very rough guidelines; what works for me. Not strict; e.g. tests for a ModelForm don't mock the model.

def add_quote(request):
    if request.method == "POST":
        form = QuoteForm(request.POST)
        if form.is_valid():
            return redirect("quote_list")
    else:
        form = QuoteForm()

    return TemplateResponse(
        request,
        "add_quote.html",
        {"form": form},
        )

This view should have 3 tests. Model/form special cases should be unit tested. And views shouldn't get much more complex.

Testing documentation

"We have always been at war with doctests"

Not entirely fair.
Doctests are great.
For testing documentation examples.

You have Sphinx docs.

Right?

You have API code examples.

In your Sphinx docs.

in any test file

def load_tests(loader, tests, ignore):
    path = os.path.join(
        settings.BASE_PATH,
        "docs",
        "examples.rst",
        )

    tests.addTests(
        doctest.DocFileSuite(path))

    return tests

Your examples are tested!

Please don't abuse this.
Keep them documentation first.

You have a setting.

ALLOW_COMMENTS, for example.

def test_comments_allowed(self):
  old_allow = settings.ALLOW_COMMENTS
  settings.ALLOW_COMMENTS = True
  try:
    # ...
  finally:
    settings.ALLOW_COMMENTS = old_allow

@override_settings(ALLOW_COMMENTS=True)
def test_comments_allowed(self):
  # ...

Testing and Django

Carl J Meyer

@carljm

carl@oddbird.net

github.com/carljm/django-testing-slides

github.com/carljm/django-testing-slides/code

Let's start a project.

I'll never get those 14 seconds back.

Not all apps are created equal.

No problem.

./manage.py test just my apps please

tests/__init__.py

Django's test discovery

It's easy to change.

settings.py

\o/

Maybe in 1.5?

Types of test

Unit tests

Integration tests

Testing models

The database makes your tests slow.

django.test.TestCase

TransactionTestCase

Just say no.

Model factories!

Using a factory

Or use factory_boy:

Why factories?

Imposing no-DB discipline.

Testing views

Unit testing views is hard.

If you unit test views

Or don't.

Integration testing views

or...

WebTest!

The markup matters.

WebTest > django.test.Client

In-browser testing

Is easier than you think.

What type of test to write?

Testing documentation

"We have always been at war with doctests"

You have Sphinx docs.

You have API code examples.

In your Sphinx docs.

in any test file

Your examples are tested!

You have a setting.

Questions?

Carl J Meyer

@carljm

carl@oddbird.net

github.com/carljm/django-testing-slides

github.com/carljm/django-testing-slides/code

`./manage.py test just my apps please`

`tests/init.py`

`settings.py`

`django.test.TestCase`

`TransactionTestCase`

Or use `factory_boy`: