Symfony 3.4 PHPUnit testing database data

Hey guys, this post is a follow up from my last phpunit in Symfony2 post. This time I’ve had to install a test environment in a Symfony 3.4 project and I’ve had some weird problems, so I thought of writing this blog post once I solved them all, it may be helpful to someone out there 🙂

Step 1 – Installing a fresh Symfony 3.4 project.

In your symfony project, run the composer require

symfony new my_project 3.4

Now run the phpunit command to run tests.

cd my_project
# if you have the phpunit phar installed globally
phpunit tests/
# if you use the phar file directly
php phpunit.phar tests/

Right off the bat, it’ll give you an error…

PHPUnit 5.7.21 by Sebastian Bergmann and contributors.

E                                                                   1 / 1 (100%)

Time: 128 ms, Memory: 14.00MB

There was 1 error:

1) Tests\AppBundle\Controller\DefaultControllerTest::testIndex
Error: Call to undefined method Symfony\Component\Yaml\Parser::parseFile()

This problem is due to an incompatibility with code dependencies from PHPUnit and Symfony 3.2+ code, specifically with the Yaml component.

Don’t panic though, PHPUnit is working to solve this issue.

More info on the PHPUnit issue here.
More info on Symfony’s explanation here.

In the meantime, we can use the PHPUnit bridge to test our application.

cd my_project
composer require --dev symfony/phpunit-bridge

Once your composer json and lock have updated, it’s time to run the test!

./vendor/bin/simple-phpunit

It works!!

Step 2 – Installing doctrine’s fixtures bundle.

First run the composer require

composer require --dev doctrine/doctrine-fixtures-bundle

Once your composer json and lock have updated, it’s time to install it in our Kernel.

// app/AppKernel.php

// ...
if (in_array($this->getEnvironment(), ['dev', 'test'], true)) {
    // ...
    $bundles[] = new Doctrine\Bundle\FixturesBundle\DoctrineFixturesBundle();
}

Now it’s installed, let’s create our first fixture. Let’s suposse we installed FOSUserBundle to manage our users, so for instance we could create a couple users like so:

// AppBundle/DataFixtures/ORM/LoadUserData.php

namespace AppBundle\DataFixtures\ORM;

use Doctrine\Common\DataFixtures\FixtureInterface;
use Doctrine\Common\DataFixtures\OrderedFixtureInterface;
use Doctrine\Common\Persistence\ObjectManager;
use Symfony\Component\DependencyInjection\ContainerAwareInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;
use AppBundle\Entity\User;

class LoadUserData implements FixtureInterface, ContainerAwareInterface
{
    private $data = [
        'ROLE_USER' => [
            'username' => 'user',
            'email' => 'user@slowcode.io',
            'plainPassword' => 'user@slowcode.io'
        ],
        'ROLE_ADMIN' => [
            'username' => 'admin',
            'email' => 'admin@slowcode.io',
            'plainPassword' => 'admin@slowcode.io'
        ],
    ];

    private $container;

    public function setContainer(ContainerInterface $container = null)
    {
        $this->container = $container;
    }

    public function load(ObjectManager $manager)
    {
        $userManager = $this->container->get('fos_user.user_manager');

        foreach ($this->data as $role => $attrs) {
            $user = $userManager->createUser();
            foreach ($attrs as $attr => $val) {
                $function = 'set'. ucwords($attr);
                $user->$function($val);
                $user->setEnabled(true);
            }
            $user->setRoles(['ROLE_USER', $role]);
            $userManager->updateUser($user, true);
        }
    }
}

Now if we run the fixtures like so

php bin/console doctrine:fixtures:load --append

We’ll get another error:

In LoadDataFixturesDoctrineCommand.php line 95:
                                                
  [InvalidArgumentException]                    
  Could not find any fixture services to load. 

To fix this, we’ll need to add this to services.yml

# services.yml
services:
    # ...
    AppBundle\DataFixtures\:
        resource: '../../src/AppBundle/DataFixtures'
        tags: ['doctrine.fixture.orm']

Yes, you guessed it, the magic here relies on the tag. Doctrine will load all services that have that ‘doctrine.fixture.orm’ tag. On the other hand, it’ll also load all the classes that implement the ORMFixtureInterface (for instance, those classes extending the Fixture class). More info on all of this in the DoctrineFixtureBundle doc page.

Step 3 – Setting a separate database for testing.

Add these lines on your config_test.yml file.

# config_test.yml
# ...
# Doctrine Configuration
doctrine:
    dbal:
        driver:   "%test_database_driver%"
        host:     "%test_database_host%"
        port:     "%test_database_port%"
        dbname:   "%test_database_name%"
        user:     "%test_database_user%"
        password: "%test_database_password%"

This will point to the new test database.

Then add the data in parameters.yml (don’t forget parameters.yml.dist for future installs)

# parameters.yml
parameters:
    # ...
    test_database_driver: pdo_mysql
    test_database_host: localhost
    test_database_port: null
    test_database_name: test_db_name
    test_database_user: test_db_user
    test_database_password: test_db_pw

Step 4 – Creating a TestCase that we will extend when ddbb fixtures are needed.

This is useful for reusing code, instead of writing it on every file. It also makes all the fixture tests more consistent.

 //tests/AppBundle/DataFixtures/DataFixtureTestCase.php
namespace Tests\AppBundle\DataFixtures;

use Doctrine\ORM\EntityManager;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Bundle\FrameworkBundle\Console\Application;
use Symfony\Component\Console\Input\StringInput;
use Symfony\Bundle\FrameworkBundle\Client;
use Symfony\Component\DependencyInjection\ContainerInterface;

class DataFixtureTestCase extends WebTestCase
{
    /** @var  Application $application */
    protected static $application;

    /** @var  Client $client */
    protected $client;
    
    /** @var  ContainerInterface $container */
    protected $container;

    /** @var  EntityManager $entityManager */
    protected $entityManager;

    /**
     * {@inheritDoc}
     */
    public function setUp()
    {
        self::runCommand('doctrine:database:drop --force');
        self::runCommand('doctrine:database:create');
        self::runCommand('doctrine:schema:create');
        self::runCommand('doctrine:fixtures:load --append --no-interaction');

        $this->client = static::createClient();
        $this->container = $this->client->getContainer();
        $this->entityManager = $this->container->get('doctrine.orm.entity_manager');

        parent::setUp();
    }

    protected static function runCommand($command)
    {
        $command = sprintf('%s --quiet', $command);

        return self::getApplication()->run(new StringInput($command));
    }

    protected static function getApplication()
    {
        if (null === self::$application) {
            $client = static::createClient();

            self::$application = new Application($client->getKernel());
            self::$application->setAutoExit(false);
        }

        return self::$application;
    }
    
    /**
     * {@inheritDoc}
     */
    protected function tearDown()
    {
        self::runCommand('doctrine:database:drop --force');

        parent::tearDown();

        $this->entityManager->close();
        $this->entityManager = null; // avoid memory leaks
    }
}

This will create the database and install the fixtures on every test that extends this TestCase. This is important as you want a consistent database with the same data every time!

Step 5 – Creating the unit, functional and integration tests

After creating your test fixtures, now you can test your services logic easily by extending the TestCase we just created, like so.

// tests/AppBundle/User/FooTest.php

class FooTest extends DataFixtureTestCase
{
    protected $fooService;

    /**
     * {@inheritDoc}
     */
    public function setUp()
    {
        parent::setUp();
        $this->fooService = $this->container->get('app.service.foo');
    }

    public function test_get_two_users()
    {
        $users = $this->entityManager->getRepository(User::class)->findAll();
        $this->assertEquals(2, count($users));
    }

  
}

Now you just need to run the phpunit command like so

./vendor/bin/simple-phpunit

I’ve created a bash wrapper script called ‘test’ that contains just this line of code so now I just need to run

bash test

Happy coding and testing! 🙂

Translatable models for translatable website in Django

This post is about using model translations (without translation files) and showing some ways to send that translations to the front end where they can be rendered.

This post is not about installing modeltranslations in Django, there’s a comprehensive guide on how to do that here.

Study case: Translatable texts that need to be editable in the django admin backend, without translation files.

Once we’ve got the modeltranslations module installed, we need to create the model we want to translate.

I’ve created the model String, but you can call it whatever you want.

#your_app/models/string.py
from django.db import models

class String(models.Model):
    key = models.CharField(max_length=80)
    value = models.CharField(max_length=255, blank=True, null=True)

    def __str__(self):
        return self.key

Now we should register our model in the translation.py, and so we can specify what field(s) will be the translatable one.

#your_app/translation.py
class StringTranslationOptions(TranslationOptions):
    fields = ('value',)


translator.register(String, StringTranslationOptions)

In a database level, this model table called your_app_string will contain the translations for every key in a column for each language: value_en, value_es, …

If you have i18n patterns in your url settings, Django will automatically know what locale to serve you through it’s own middleware system. For more info on this, there’s a comprehensive guide here.

Otherwise, you can set the locale manually through the request, up to you. More info on this here.

Once Django already knows what’s the current locale, we just need to get the strings from the database. The modeltranslations module will automatically give us the value translated to the current locale.

Translating the front end

We are able to transfer the strings to the frontend templates through many ways. For instance, we could load the strings on every view before rendering the template. However, this is bit of a code smell, as code is too coupled and it’s not very mantainable. We can do better like so:

1. Using a Context Processor

#your_app/context_processors/trans.py
from your_app.models import String


def trans(request):
    trans = {}
    for string in String.objects.all():
        trans[string.key] = string.value

    return {'trans': trans}

Your’ll need to register the context processor in settings.py

#settings.py
TEMPLATES = [
    {	
    	'...',
        'OPTIONS': {
            'context_processors': [
            	'...',
                'your_app.context_processors.trans',
            ],
        },
    },
]

What this will do is, after every render function in our views, it’ll include the array of all translations. That’s why you don’t need to load any extra file in the template.

Now in the front end, you can use it like so:

{{ trans.hello_world }}

{{ trans.hello_world_wrong_key }}

This is similar to loading the strings just before every render in the views, however we’ve removed the coupling by centralizing the code in one place.

The main problem though is that if there’s a key that’s not included, Django will just ignore that key, not giving us any feedback if a translation is missing. This is annoying if we’re developing, as we won’t catch missing translations for certain keys.

2. Using a filter (templatetag)

#your_app/templatetags/trans.py
@register.filter(name='trans')
def trans(field):

    translations = {}
    for string in String.objects.all():
        translations[string.key] = string.value

    try:
        trans = translations[field.__str__()]
    except KeyError:
        trans = '[*' + field + '*]'

    return trans

Now in the front end, you can use it like so. This time, different from context_processor, we won’t need to register it in the settings.py but we do need to load it in the template.

{% load trans %}

{{ 'hello_world' | trans }}

{{ 'hello_world_wrong_key' | trans }}

This one gives feedback if the key doesn’t exist, but as you can see, it loads all the strings every time it finds a key to be rendered. This is very inefficient!

3. Using a simple tag (templatetag)

Very similar to the filter, but there’s a suttle difference, this one can load the context.

#your_app/templatetags/trans.py
@register.simple_tag(name='trans', takes_context=True)
def trans(context, field):
    translations = context.request.session.get('translations')

    if not translations:
        translations = {}
        for string in String.objects.all():
            translations[string.key] = string.value
        context.request.session['translations'] = translations

    try:
        trans = translations[field.__str__()]
    except KeyError:
        trans = '[*' + field + '*]'

    return trans

Loading the context allows us to save data in memory, so it doesn’t have to load the translations from the database every single time there’s a string to be translated.

In the front end we still need to load the tag.

{% load trans %}

{% trans 'hello_world' %}

{% trans 'hello_world_wrong_key' %}

This one is not only clean and efficient, it also gives feedback if a translation key doesn’t exist in the database. Furthermore, since we now have the context (it can’t be loaded with a filter), the translations will be loaded from the ddbb just once (when it tries to translate the first key). When there’s a second key to be translated, it’ll get the array of translations from the context and just give the value translated, or the key if it can’t find it.

So there you have it, three different ways to present the translations in the front end with a custom model translation.

Other useful resources:

Django translations
Django model translations