Symfony2 PHPUnit testing database data

Today I had to test some complex algorithm logic we’re writing for a client app. The algorithm I want to test interacts with the DDBB several times through query builders and repositories, so it’s necessary to test this database access, as we need to know the sql queries give the right data also.

It’s the first time I set up a test environment (I worked with Unit Testing before, but never set up the env from scratch). So after reading a bit of doc from the internet, I set up the unit test environment, then wrote some tests that interacted with a test database and test fixtures.

I’ve put together a bit of guide for anyone wanting to get started on this.

Step 1 – Installing doctrine’s fixtures bundle.

First run the composer require

composer require --dev doctrine/doctrine-fixtures-bundle

Once your composer json and lock have updated, it’s time to install it in our Kernel.

// app/AppKernel.php

// ...
if (in_array($this->getEnvironment(), ['dev', 'test'], true)) {
    // ...
    $bundles[] = new Doctrine\Bundle\FixturesBundle\DoctrineFixturesBundle();
}

Step 2 – Setting a separate database for testing.

Add these lines on your config_test.yml file.

#config_test.yml

...

# Doctrine Configuration
doctrine:
    dbal:
        driver:   "%test_database_driver%"
        host:     "%test_database_host%"
        port:     "%test_database_port%"
        dbname:   "%test_database_name%"
        user:     "%test_database_user%"
        password: "%test_database_password%"
        # Workaround for DBAL 2.5 auto-detect -> with server_version allows to doctrine:database:create
        server_version: 5.6 # your database server version here

This will point to the new test database.

Then add the data in parameters.yml (don’t forget parameters.yml.dist for future installs)


# This file is auto-generated during the composer install
parameters:

    ...
    test_database_driver: pdo_mysql
    test_database_host: localhost
    test_database_port: null
    test_database_name: test_db_name
    test_database_user: test_db_user
    test_database_password: test_db_pw

Step 3 – Creating a TestCase that we will extend when ddbb fixtures are needed.

This is useful for reusing code, instead of writing it on every file. It also makes all the fixture tests more consistent.


namespace Tests\AppBundle\DataFixtures;

use Doctrine\ORM\EntityManager;
use Symfony\Bundle\FrameworkBundle\Test\WebTestCase;
use Symfony\Bundle\FrameworkBundle\Console\Application;
use Symfony\Component\Console\Input\StringInput;
use Symfony\Bundle\FrameworkBundle\Client;
use Symfony\Component\DependencyInjection\ContainerInterface;

class DataFixtureTestCase extends WebTestCase
{
    /** @var  Application $application */
    protected static $application;

    /** @var  Client $client */
    protected $client;
    
    /** @var  ContainerInterface $container */
    protected $container;

    /** @var  EntityManager $entityManager */
    protected $entityManager;

    /**
     * {@inheritDoc}
     */
    public function setUp()
    {
        self::runCommand('doctrine:database:drop --force');
        self::runCommand('doctrine:database:create');
        self::runCommand('doctrine:schema:create');
        self::runCommand('doctrine:fixtures:load --append --no-interaction --fixtures=tests/AppBundle/DataFixtures/ORM');

        $this->client = static::createClient();
        $this->container = $this->client->getContainer();
        $this->entityManager = $this->container->get('doctrine.orm.entity_manager');

        parent::setUp();
    }

    protected static function runCommand($command)
    {
        $command = sprintf('%s --quiet', $command);

        return self::getApplication()->run(new StringInput($command));
    }

    protected static function getApplication()
    {
        if (null === self::$application) {
            $client = static::createClient();

            self::$application = new Application($client->getKernel());
            self::$application->setAutoExit(false);
        }

        return self::$application;
    }
    
    /**
     * {@inheritDoc}
     */
    protected function tearDown()
    {
        self::runCommand('doctrine:database:drop --force');

        parent::tearDown();

        $this->entityManager->close();
        $this->entityManager = null; // avoid memory leaks
    }
}

Most of the code is taken from StackOverflow.

This will create the database and install the fixtures on every test that extends this TestCase. This is important as you want a consistent database with the same data every time!

It’s also important to pass the parameter

--fixtures=tests/AppBundle/DataFixtures/ORM'

This way you can set up all your test fixtures separate from your app fixtures, which will stay clean.

Step 4 – Creating the unit, functional and integration tests

After creating your test fixtures, now you can test your services logic easily by extending the TestCase we just created, like so.

class FooTest extends DataFixtureTestCase
{
    protected $fooService;

    /**
     * {@inheritDoc}
     */
    public function setUp()
    {
        parent::setUp();
        $this->fooService = $this->container->get('app.service.foo');
    }

    public function testFooTrue()
    {

        ... 
        //custom logic
        ... 

        $this->assertEquals(true, $value);
    }

  
}

Now you just need to run the phpunit command like so

phpunit tests/

And that’s all you need! Happy testing! 🙂

PS: This is a very easy way to get started. For more complex testing environment settings, you can try LiipFunctionalTestBundle

Asking the right questions before building an MVP

This last week we’ve had two potential new clients ask us for an MVP RFQ (Request for quotation).

MVP means Minimal Viable Product, which in the software world means a very simple app that shows your future potential app in a simplified way.

An MVP is normally used to asses future app behaviour, get feedback from users and work from there, and also as a showcase for future investors – investors very rarely invest in ideas, they prefer to see something functional, with customer response and feedback if possible.

It is important that the MVP shows at least a very important feature of your future app. For example, for an ecommerce MPV, logging in a user is not as important as being able to buy a product.

So back to the story, these two clients contacted us and told us what they wanted in a very different way. It became very clear to me that client A knew exactly what he wanted, whilst client B did not. Client A gave direct answers, client B did not.

If a client doesn’t really know what he wants, asking for information becomes a hurdle, ideas seem to jump all around while nothing ever gets done. If you relate to this, you know how frustrating it is… specially because no one is paying for your time at this stage!!

So what’s really important at that moment is to ask the right questions. I cannot stress enough how important this is, because it sets the pace for the whole project.

For example,

Summarize your app in one sentence.
There is a reason why this is number one. This is the most important question – of course they can use more than one sentence, but asking them this puts them in a place where they have to really think of the most important element of the app, not just random features they’re going to add. This is key to understanding properly what your client wants.

What is the reason this app exists?
Spot the USP ‘Unique Selling Point’. There might be none, and this is also fine.

What is the budget for the app and for the MVP?
This will provide you with information of how big this app is going to be, at least at the beginning. The budget might not be a number, it might be a range. This is fine also, because then you’re able to scope the size of the whole project.

How will you make money from the app?
Sometimes clients think that we, as developers, don’t need this question answered, specially at an MVP level. I think it’s useful to know where the focus and priorities are going to be, so you can start thinking big and preparing for the future.

These are some of my non technical preferred questions. Believe me, they do the job right! Once, and only once, you get answers for these, you can start asking the technical ones – maybe that’s for a future post 🙂

Deploy on OVH shared hosting with different PHP versions

Recently we’ve been working on a Symfony project that needed to be deployed on a shared hosting by OVH.

This server was running PHP 5.4 and we could not upgrade it because they had some legacy projects that needed to remain..

So after banging our head quite a few times, we found out that you could create a configuration file so that OVH server reads and reconfigures the settings of the server for that project/folder only.

The file must be called .ovhconfig and it must be on the target folder the DNS points to, so in our case being a Symfony project it should be in the /web folder

app.engine=php
app.engine.version=5.6
http.firewall=none
environment=production
container.image=stable

PHP 5.6 and PHP 7 on same Mac with Liip

So the other day I wanted to upgrade to php7 but didn’t want to completely remove php56 as I still have legacy projects to maintain and I’m not ‘dockerized’ yet..

You can install php with liip here.

Liip doesn’t overwrite Apple’s php binaries, it installs it under a php5 folder under /usr/local/php5 and then creates the link.

All you now need is this handy bash script to switch from one php version to the other.

#!/bin/bash

#usage:
#./switch-php.sh 
#./switch-php.sh php5
#./switch-php.sh php7

#activate php56
if [ "$1" == "php5" ]; then
   sudo rm /usr/local/php5
   sudo ln -s /usr/local/php5-5.6.29-20170114-210819 /usr/local/php5
   sudo pkill php-fpm && sudo php-fpm
   echo "activated php5;"
   sudo apachectl restart
elif [ "$1" == "php7" ]; then
   sudo rm /usr/local/php5
   sudo ln -s /usr/local/php5-7.1.9-20170914-100859 /usr/local/php5
   sudo pkill php-fpm && sudo php-fpm
   echo "activated php7;"
   sudo apachectl restart
else
   echo "parameter expected: php5 | php7"
fi

PS: You’ll have to obviously adapt the version to the one you installed!

Bundle-less applications with Symfony Flex and Symfony4

So these last few days we’ve had a pretty interesting thread on twitter with @gigo6000 and @WladimirAvila about Symfony’s new directory structure, specially about Bundle Inheritance. You can check the thread here

For the ones like me that still didn’t know or haven’t read it somewhere, bundle inheritance is deprecated from Symfony 3.4 onwards, therefore it makes sense to have bundle-less applications from now on.

It turns out that bundle inheritance was rarely used anyway, except for customizing some templates. Who hasn’t inherited the FOSUserBundle before customizing just the login page? And maybe the entity for some extra attributes?

The question that I ask myself though is how are we going to customize all of this now? Not just templates, but all the config associated, services, entities, forms…

To me, one of the best things about Symfony’s magic was the ability to extend ready made bundles/vendors. Hope there’s still a way to do this!

More info on the Symfony4 directory structure here.

More info on bundle inheritence here

Gedmo Tree in Symfony3

So today at work I’ve had to rethink the way we work with products in an app, and it turns out the best way to face the problem is having a tree based architecture, as products can have sub-products and so on and so forth… so instead of reinventing the wheel, why not use a well established library that uses trees right? Gedmo please!

So it’s actually really easy to install Gedmo extensions to your project.

1) First of all composer install it

php -d memory_limit=-1 /usr/local/bin/composer require stof/doctrine-extensions-bundle

(yes I don’t like playing with my php.ini so for this command I rather pass the memory_limit parameter, also remember to put your composer path correctly, as it won’t read your alias if you pass parameters to php)

2) After that you need to include it to you Kernel

$bundles = [
			...
            // Gedmo
            new Stof\DoctrineExtensionsBundle\StofDoctrineExtensionsBundle(),
            ...
        ];

3) And the final step is adding the extra configuration in your config.yml

Under the doctrine.orm configuration:

# Doctrine Configuration
doctrine:
    ...

    orm:
        auto_generate_proxy_classes: "%kernel.debug%"
#        naming_strategy: doctrine.orm.naming_strategy.underscore
#        auto_mapping: true

        default_entity_manager: default
        entity_managers:
            default:
              connection: default
              naming_strategy: doctrine.orm.naming_strategy.underscore
              auto_mapping: true
              dql:
                  numeric_functions:
                      ACOS: DoctrineExtensions\Query\Mysql\Acos
                      COS: DoctrineExtensions\Query\Mysql\Cos
                      RADIANS: DoctrineExtensions\Query\Mysql\Radians
                      SIN: DoctrineExtensions\Query\Mysql\Sin
              mappings:
                  gedmo_translatable:
                      type: annotation
                      prefix: Gedmo\Translatable\Entity
                      dir: "%kernel.root_dir%/../vendor/gedmo/doctrine-extensions/lib/Gedmo/Translatable/Entity"
                      alias: GedmoTranslatable # (optional) it will default to the name set for the mapping
                      is_bundle: false
                  gedmo_translator:
                      type: annotation
                      prefix: Gedmo\Translator\Entity
                      dir: "%kernel.root_dir%/../vendor/gedmo/doctrine-extensions/lib/Gedmo/Translator/Entity"
                      alias: GedmoTranslator # (optional) it will default to the name set for the mapping
                      is_bundle: false
                  gedmo_loggable:
                      type: annotation
                      prefix: Gedmo\Loggable\Entity
                      dir: "%kernel.root_dir%/../vendor/gedmo/doctrine-extensions/lib/Gedmo/Loggable/Entity"
                      alias: GedmoLoggable # (optional) it will default to the name set for the mappingmapping
                      is_bundle: false
                  gedmo_tree:
                      type: annotation
                      prefix: Gedmo\Tree\Entity
                      dir: "%kernel.root_dir%/../vendor/gedmo/doctrine-extensions/lib/Gedmo/Tree/Entity"
                      alias: GedmoTree # (optional) it will default to the name set for the mapping
                      is_bundle: false

Yes, it’s important to remove the auto_mapping as it won’t recognize the key under doctrine.orm

And under the stof_doctrine_extensions configuration:

# Stof Configuration
stof_doctrine_extensions:
    default_locale: "%locale%"
    translation_fallback: true
    persist_default_translation: true

    # Only used if you activated the Uploadable extension
    uploadable:
        #stof_doctrine_extensions.uploadable.validate_writable_directory
        validate_writable_directory: true

        # Default file path: This is one of the three ways you can configure the path for the Uploadable extension
        default_file_path:       "%kernel.root_dir%/../web/uploads"

        # Mime type guesser class: Optional. By default, we provide an adapter for the one present in the HttpFoundation component of Symfony
        mime_type_guesser_class: Stof\DoctrineExtensionsBundle\Uploadable\MimeTypeGuesserAdapter

        # Default file info class implementing FileInfoInterface: Optional. By default we provide a class which is prepared to receive an UploadedFile instance.
        default_file_info_class: Stof\DoctrineExtensionsBundle\Uploadable\UploadedFileInfo
    orm:
        default:
            translatable:   true
            blameable:      false
            timestampable:  true
            tree:           true
            uploadable:     false
            sluggable:      true

That’s it! You’re set, super easy right?

Now let’s go to the entity, in my case it’s the Product. Don’t forget to import the Gedmo annotation and then add the annotation to the class, like so:

use Gedmo\Mapping\Annotation as Gedmo;

/**
 * @ORM\Entity(repositoryClass="AppBundle\Repository\ProductRepository")
 * ...
 * @Gedmo\Tree(type="nested")
 */
class Product
{
	/**
     * @Gedmo\TreeLeft
     * @ORM\Column(name="lft", type="integer")
     */
    private $lft;

    /**
     * @Gedmo\TreeLevel
     * @ORM\Column(name="lvl", type="integer")
     */
    private $lvl;

    /**
     * @Gedmo\TreeRight
     * @ORM\Column(name="rgt", type="integer")
     */
    private $rgt;

    /**
     * @Gedmo\TreeRoot
     * @ORM\ManyToOne(targetEntity="Product")
     * @ORM\JoinColumn(name="tree_root", referencedColumnName="id", onDelete="CASCADE")
     */
    private $root;

    /**
     * @Gedmo\TreeParent
     * @ORM\ManyToOne(targetEntity="Product", inversedBy="children")
     * @ORM\JoinColumn(name="parent_id", referencedColumnName="id", onDelete="CASCADE")
     */
    private $parent;

    /**
     * @ORM\OneToMany(targetEntity="Product", mappedBy="parent")
     * @ORM\OrderBy({"lft" = "ASC"})
     */
    private $children;

    ...
}

Running the command

bin/console doc:gen:entities AppBundle:Product

will generate all the getters and setters, pretty handy…

And now the repository:

use Gedmo\Tree\Entity\Repository\NestedTreeRepository;

/**
 * ProductRepository
 *
 * This class was generated by the Doctrine ORM. Add your own custom
 * repository methods below.
 */
class ProductRepository extends NestedTreeRepository
{
...
}

The NestedTreeRepository has very useful functions, play with them before making your custom ones, cause they’ve already thought with pretty much everything, for instance the function childrenHierarchy() gives you the nested array, with decoration output if you pass parameters as options. You can find more info in the RepositoryInterface and RepositoryUtilsInterface from Gedmo\Tree namespace.

PS: Don’t forget to run your schema update command

bin/console doc:sch:update --force

Now you focus on what really matters, you project’s logic.

Happy coding! 🙂