PHP Symfony versioning and composer

Lately I’ve been switching to new versions and upgrading Symfony for client projects that I cannot reveal, but what I would like to share in this simple and short post is what combinations worked for me after many hours switching versions and dependancies.

Symfony 4

{
    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "sensio/framework-extra-bundle": "^5.4",
        "symfony/asset": "4.3.*",
        "symfony/console": "4.3.*",
        "symfony/dotenv": "4.3.*",
        "symfony/expression-language": "4.3.*",
        "symfony/flex": "^1.3.1",
        "symfony/form": "4.3.*",
        "symfony/framework-bundle": "4.3.*",
        "symfony/http-client": "4.3.*",
        "symfony/intl": "4.3.*",
        "symfony/monolog-bundle": "^3.1",
        "symfony/orm-pack": "*",
        "symfony/process": "4.3.*",
        "symfony/security-bundle": "4.3.*",
        "symfony/serializer-pack": "*",
        "symfony/swiftmailer-bundle": "^3.2",
        "symfony/translation": "4.3.*",
        "symfony/twig-bundle": "4.3.*",
        "symfony/validator": "4.3.*",
        "symfony/web-link": "4.3.*",
        "symfony/webpack-encore-bundle": "^1.6",
        "symfony/yaml": "4.3.*",
        "jms/serializer-bundle": "3.4.1",
        "friendsofsymfony/user-bundle": "2.1.2"
    }
}

This is the easiest one since is the last version and you can just install it following the docs. Added some nice packages that I use always just to JSON serialize entities.

Symfony 3.4

{
"require" : {
        "php" : "^7.1",
        "symfony/symfony" : "3.4.x-dev",
        "symfony/webpack-encore-bundle" : "~1.6",
        "symfony/swiftmailer-bundle" : "3.3.*",
        "symfony/monolog-bundle" : "3.4.0",
        "symfony/console" : "^3.4",
        "symfony/serializer": "^3.4",
        "sensio/distribution-bundle" : "5.0.25",
        "sensio/framework-extra-bundle" : "5.4.0",
        "doctrine/orm": "2.6.4",
        "doctrine/doctrine-bundle": "2.0.x-dev",
        "twig/extensions" : "1.5.4"
    }
}

Basic install. This where you would like to go in case you are still using 2.8 version that will be soon out of maintenance

Symfony 3.3

{
"require": {
        "php": ">=5.6.27",
        "symfony/symfony": "^3.3",
        "doctrine/orm": "^2",
        "doctrine/doctrine-bundle": "1.10.0",
        "symfony/swiftmailer-bundle": "~2",
        "symfony/monolog-bundle": "^3",
        "sensio/distribution-bundle": "~5.0",
        "sensio/framework-extra-bundle": "^3",
        "incenteev/composer-parameter-handler": "~2.0",
        "knplabs/knp-snappy-bundle": "~1.4",
        "knplabs/doctrine-behaviors": "~1.4",
        "friendsofsymfony/user-bundle": "~2.0@dev",
        "doctrine/doctrine-fixtures-bundle": "^2.3",
        "twig/extensions": "^1.4",
        "mopa/bootstrap-bundle": "dev-master",
        "m6web/guzzle-http-bundle": "~2.0",
        "doctrine/migrations": "^1.5",
        "doctrine/doctrine-migrations-bundle": "^1.3",
        "jms/serializer-bundle": "1.5.0"
    }
}

The important difference in this 3.3 version is that it still runs on PHP 5.6+ but if you can it would be the best to go to 3.4 or even 4 but that requires a big refatoring in many corners.

Postman self documenting tests and runner import

I’ve been working on the last days doing extensive API testing and needed to find an easy way to document my tests. Postman offers already something very useful that uploads your tests documentation to their website. But sometimes we need just a simple HTML that can be privately delivered to the client.

That’s where this project was born:

https://github.com/martinberlin/postman-reporter

The intention of this simple PHP script is to generate a Standalone HTML for your Postman tests that you can send to the client without the need to upload all the tests in the open internet.

It serves to achieve two things:

  1. Make a standalone HTML document from your Postman Collections
  2. Import the test run-results into a mysql database

With the second one only the importing is done. It’s then up to you how to present this data. It populates two tables, resume and detail, first one with the performance result and the detailed with a line per test. Much more information can be extracted from the runner json this is just a kickstart idea. Have fun with it!

If it calls your interest then please read more details in the github repository.

How routing internally works in Symfony 4 framework

At the moment of writing this I’m testing the Symfony 4 Framework after reading the following article and also Fabian’s latest posts on Medium.  I’m testing weather the not yet officially released version, will be a future candidate for some new project or not. But the interesting part of this research is that I already realized what I won’t do in new projects. But let’s focus first on the mission of describing how routing works internally in the framework.
So we will go from the Frontend Controller up to the internal guts that say from the route /test let’s execute Controller:test()

http:// bestpractices.local/test

The front-end controller located in /web/index.php bootstraps the framework:

$kernel = new Kernel(getenv('APP_ENV'), getenv('APP_DEBUG'));
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();

As we already know all this framework is based on Request -> Response concept from the beginning to the latest controller action and the Kernel is no exception.  If we open this Kernel we can see that it’s mission is to setup Cache and Logs directories, register the bundles, configure the Container and Routing.

To go deeper in the chain we need to open HttpKernel with the mission described as:
HttpKernel notifies events to convert a Request object to a Response one.
There on the handleRaw method “Handles a request to convert it to a response” is where the whole framework workflow takes place:

namespace Symfony\Component\HttpKernel;

class HttpKernel implements HttpKernelInterface, TerminableInterface {
private function handleRaw(Request $request, $type = self::MASTER_REQUEST)
    {
        $this->requestStack->push($request);
        // request
        $event = new GetResponseEvent($this, $request, $type);
        $this->dispatcher->dispatch(KernelEvents::REQUEST, $event);

  // At this point the routing is already resolved
  // dump($request->attributes);exit();

        if ($event->hasResponse()) {
            return $this->filterResponse($event->getResponse(), $request, $type);
        }

        // load controller
        if (false === $controller = $this->resolver->getController($request)) {
            throw new NotFoundHttpException(sprintf('Unable to find the controller for path "%s". The route is wrongly configured.', $request->getPathInfo()));
        }

// ... Return a response

As we can see on the first lines, the Routing part, is resolved dispatching an Event called KernelEvents::REQUEST

namespace Symfony\Component\HttpKernel\EventListener;

class RouterListener implements EventSubscriberInterface {

public function onKernelRequest(GetResponseEvent $event)
    {
        $request = $event->getRequest();

        $this->setCurrentRequest($request);

        if ($request->attributes->has('_controller')) {
            // routing is already done
            return;
        }

        // add attributes based on the request (routing)
        try {
            // matching a request is more powerful than matching a URL path + context, so try that first
            if ($this->matcher instanceof RequestMatcherInterface) {
                $parameters = $this->matcher->matchRequest($request);
                // dump($parameters); // here to see it matched

            } else {
                $parameters = $this->matcher->match($request->getPathInfo());

            }

And in this part of the code we can see that dumping the $parameters the Request attributes are already assigned

$parameters = array(
  "_controller" => "App\Controller\DefaultController::testAction"
  "_route" => "test"
);

Note: Request->attributes is used to store information about the current Request such as the matched route, the controller, etc
But more interesting is to replace this dump for a : dump($this->matcher);
Then we can see that the real matcher of the routing is done by srcDevDebugProjectContainerUrlMatcher since we are currently using DEV as an environment.

This file, that we can find in this path /var/cache/dev/srcDevDebugProjectContainerUrlMatcher.php , is the responsible of returning the request parameters with the right controller to be executed for this route.
Since doing all the routing on the fly will be a real show stopper and we are concerned with performance: reading the configuration from the filesystem may slow down the application.

That’s why there’s a PhpMatcherDumper class which can generate an implementation of UrlMatcherInterface with all configuration in an optimized way.

Even in DEV, this UrlMatcher file, is only rendered again if we update any part of the routing. That’s one of the reasons clearing all cache will make a slow first request.

Lessons learnt from this research are:

  • Keep your routes clean
    Not used / deprecated routes that are still there will make this UrlMatcher unnecessary long and hence slower to resolve.
  • Avoid at all costs using Annotations for the routing
    It’s generally considered a bad practice in OOP and contains much more Cons than Pros
  • Avoid using Assetic
    This generates fully unnecessary routes for every assset in your HTML templates. New versions of Symfony will not come with Assetic so it’s better to get along without it.

I never care to check this routing to the core before and I highly recommend from time to time to check the internals of Symfony. You will learn a lot of the framework essentials and also will be more cautious of how to take the best of it. In my personal case, I learnt much more doing this small research taking your own conclusions, than reading some other developer top 5 list on framework optimization.

The code to reproduce this can be downloaded from public repository:
https://github.com/martinberlin/symfony-flex

Integrating SSO in a project using Shibboleth authentication

Shibboleth is an open-source identity federation solution that’s used in research and education communities worldwide.

Graphic by ncsu.edu

Basically is a federating agent that consists of 2 components:

1- Service Provider (SP)
 which request attributes, represented in this implementation als storage.luckycloud.de using Apache2-mod-shib 
2- Identity Provider(IDP)
 which broadcasts attributes, and will be implemented in the Login-logic of the site

Now the SP is actually out-of-the-box with apache2 module mod-shib. But my interest here is to do the Identity Provider part als Symfony module.

UPDATE: The Shibboleth take on this was deprecated and we installed with Luc an SSO-version since Seafile now support OAUTH2 authentication. The changes are already implemented and tested on live:

https://storage.luckycloud.de

Because of security I cannot give any details of this implementation I would like to comment that is working without issues since day zero and it’s saving a lot of time preventing double login and I hope enhancing user experience.

Tippgeschwindigkeit Composer (Making it faster)

Nach letzes composer update, ich kriege immer diese Meldung nach update /install : “You are running composer with xdebug enabled. This has a major impact on performance and will break your balls”
So ich habe die richtige Lösung gefunden und wurde mit euch teilen:
$ php -i | xdebug

Guck mal welche config ist schuldig, in meine Ubuntu ist:
/etc/php5/cli/conf-d/20-xdebug.ini

Und einfach comment:
;zend_extension=xdebug.so

Voila! Deine composer is viel schneller ab jetzt. Für mich ist ungefähr 3x schneller.

Good architecture and why not to render Controllers inside Views

In Symfony, Zend and other frameworks you can rend a controller inside a view like this:

{{ render(controller('AcmeArticleBundle:Article:recentArticles', { 'max': 3 })) }}

 Creating and Using templates documentation here.

Just because there is a possibility to render(ControllerMethod,{ params}) in a template doesn’t mean you have to use it.

Doing so leads almost always to a shitty architecture, the turning point where projects start to be hard to debug, since you are mixing a VIEW (Presentation layer) with a CONTROLLER, that in turn renders another VIEW. You get the point.

What happens afterwars is a little bit counter-productive when debugging:
An error in the ControllerMethod is rendered as a template View error.   Not so nice isn’t it ?

Controller methods should Return a View that has all the required parameters to render the View. But if inside that View, you are going around the loop, rendering another Controller that in turn has another View, can be the start of a infinite loop that swallows the universe and you should get a warning red light in your Frontal Lobe. Overcomplicating things with quick and dirty solutions is the wrong way in Software architecture and the responsible of a big amount of relationship problems.

I vouch for strong architecture in software projects

This cheap solutions, like using this render commodities, lead to the start of the bad. And I suggest to avoid it as much as you can unless there is no other possible way.  And certainly there is!

That is the reason to use MVC. To separate Code from Presentation layer, start mixing both, and your architecture will leak.

Symfony Bundles that come always handy for Admin Panels

After a quite accidental start with the Framework I’m enjoying a lot developing stuff with their Form component.
Using bootstrap and almost without any front-end javascript one can build great Admin-dashboard forms, using only a :
{{ form_start(form) }}
{{ form_end(form) }}

Where you want the form to be rendered in the twig template.

As one of my jobs is building friendly Admin Panels, I started to realize that apart of some custom tailored-made solutions, frecuently there is a ground base that works for almost any CMS or CRUD (CReate, Update , Delete manager)

So instead of googling and fetching them one by one, you can find them, or comment them in this post:

  1. TinymceBundle
    Great one.  After installing it, you have a ready made
    {{ tinymce_init() }}
    to put at the end of your twig template. All what you need to do is adjust the config and add a “tinymce” class in your FormTypes.
    Alone with that you can build a CMS in some minutes.
    How cool is that ?
  2. KnpPaginatorBundle
    Whenever we list stuff and becomes large, we need to paginate. This one comes really handy. And like almost all the KNP bundles, is a really nice piece of code, and developer friendly.
  3. Bootstrap-bundle
    Bootstrap is the state of the art front-end to deploy Admin Dashboards.  I had a little fight before making this one work in my Linux dev machine, but is just because I’m not a very front-end guy at the moment, and didn’t had this CSS tools to compile things.  Actually you don’t need this if you want bootstrap base, but I though that having it in a Bundle and thus, being able to install it by composer was a cool thing. And it is!

And that’s basically the 3 things I mostly use all the time.
There are hundreds more, but the basic, ground structur is there.  Making a login / password with FOS User Bundle is also very easy.

In the beginning, the first 2 months or so I was looking towards something that creates an Automatic CRUD manager from a mysql table. After learning more Symfony, I realized that you don’t really need something like that, when it’s quite easy to create your Entities from an existing mysql DB and after that make some Form types and base templates. Cannot get more simplified!
Of course will take some effort, but at the end, you have a custom tailored panel with validation at the top where you can add the customization on top to make your Dashboard more usable.
Hopefully you will make your client happier as well.

Composer lines to add this:

require: {
...
"stfalcon/tinymce-bundle": "dev-master",
"knplabs/knp-paginator-bundle": "dev-master",
"braincrafted/bootstrap-bundle": "~2.0",
"twbs/bootstrap": "3.0.*",
"jquery/jquery":  "2.1.3"
}

Symfony2 Best Practices resume

Today I come across this Best practices in Symfony PDF:
http://symfony.com/doc/download-best-practices-book/

And I though about my own practices and where to apply some of this seen in the Best practices book.

Here is a brief 3 points summary of things I would like to change in my next projects:

  1. Using Constant instead of variables in parameters.yml for those settings that rarely (or never) change
    // src/AppBundle/Entity/Post.php

    namespace AppBundle\Entity;
    class Post {
    const NUM_ITEMS = 10;

    }
    As pointed  in the book, doing so, the benefit is that you can access this constant from almost anywhere in your application. Including your twig templates. This is exactly the part I was not aware of:
    // src/AppBundle/Resources/views/Controller/mytwigtemplate.html.twig

    Displaying the {{ constant (‘NUM_ITEMS’ , post ) }} most recent results.
  2. Keep your controllers Thin

    Symfony follows the philosophy of “thin controllers and fat models”. This means that controllers should hold just the thin layer of glue-code needed to coordinate the different parts of the application. As a rule of thumb, you should follow the 5-10-20 rule, where controllers should only define 5 variablesor less, contain 10 actions or less and include 20 lines of code or less in each action. This isn’t an exact science, but it should help you realize when code should be refactored out of the controller and into a service.

    That’s an easy to remember Rule:
    5   Variables or less
    10 Actions (Calls to services / Forms validation, etc)
    20 Lines of code

    Storing hundred of lines for an Action is always a bad idea specially if you are not the only one working for the project.  And it mostly denotes some bad architectural start, since I can imagine that a controller Action has so many lines has parts that are common and can be reused in another actions. So why not to put all related actions in a service ?

    In this same Chapter, has a best practice that I consider really important, and is in my point of view how an ideal Symfony oriented project should be thought:
    You should aggressively decouple your business logic from the framework while, at the same time, aggressively coupling your controllers and routing to the framework in order to get the
    most out of Symfony

  3. Use eventListeners.
    As an example: Pre and Post Hooks
    If you need to execute some code before or after the execution of your controllers, you can use the EventDispatcher component to set up before/after filters

I find eventListeners one of the most cooles features. They are many use-cases, for example OneUp Uploader bundle uses it to handle File uploads, calling an event listener each time a file is uploaded through Ajax. Like that you can find many real use cases to do this kind of automatic things in the background.

If you like any other interesting part of this Symfony Best practices Book, feel free to join the conversation

Crud in 5 minutes per Table using Sonata and Doctrine ORM

Create, read, update and delete is something you need to manage in any Project.

UPDATE: I took the decision of taking down the sonata example, since I consider now a much better way to do this with Symfony directly without using Sonata at all. To see an example of a custom admin in action please refer to https://github.com/martinberlin/supercomprador that is the admin backend for my project supercomprador.es

Here is simple formula that works using this simplified version of Sonata Admin and here is the list of steps to achieve a new Crud list per entity:

  1. Design your table structure or pick an existing one
  2. Use Doctrine to create your Entity Classes from the Tables
  3. Add an admin service in Sonata to bring up the Crud functionality

Expanding the points above:

#2 Converting existing mySql tables in Doctrine Entities
After adding your mySql database credentials to app/config.yml, having existing table structures / data, run the following command in the project root:
php app/console doctrine:mapping:import –force yourSymfonyBundle xml

This will generate ORM mapping schema information as xml in the directory Resources/config/doctrine that will be used with the next command to generate the Doctrine entities in the Synfony Entity folder.

This is to create the Tables when you have the entities written:
php app/console doctrine:schema:create

For more information read the documentation regarding Symfony and Doctrine

#3 Adding the Entities in Sonata to enable the Crud functionality

<?php
namespace BaseAdmin\BaseAdminBundle\Admin;

use Sonata\AdminBundle\Admin\Admin;
use Sonata\AdminBundle\Datagrid\ListMapper;
use Sonata\AdminBundle\Datagrid\DatagridMapper;
use Sonata\AdminBundle\Form\FormMapper;

class CityAdmin extends Admin
{
    // Fields to be shown on create/edit forms
    protected function configureFormFields(FormMapper $formMapper)
    {
        $formMapper
            ->add('city', 'text', array('label' => 'City'))
            ->add('fmuser')
            ->add('name') //if no type is specified, SonataAdminBundle tries to guess it
        ;
    }

    // Fields to be shown on Filter forms (right)
    protected function configureDatagridFilters(DatagridMapper $datagridMapper)
    {
        $datagridMapper
            ->add('city')
            ->add('fmuser')
        ;
    }

    // Fields to be shown on Datagrid lists
    protected function configureListFields(ListMapper $listMapper)
    {
        $listMapper
            ->addIdentifier('city')
            ->add('name')
            ->add('fmuser')
        ;
    }
}

Screenshots:
basicAdmin Entities list

basicAdmin Crud List

The code to achieve this is Open source and the repository is located here:

https://github.com/martinberlin/SymfonyBaseAdmin

OOP Fundamentals

There are many different takes on object oriented programming. I’ve saw this video of “Uncle Bob” latest week and I wanted to write a post about the fundamentals of OOP programming.

I like the way he explains OOP, interfaces, polymorphism and other related areas of object oriented methodology. So before starting with my post series of Symfony, I wanted to review a couple of basics, and how they are supposed to be used.

The benefits of working using OOP is that the application itself becames minimal, and the modules can be coded in independant pieces, that have a decoupled arquitecture and are independantly deployed, hence repairing a certain part does not brake the others. In Symfony this modules are called bundles. A good programming example, is the Symfony components itself, and another Business Logic example I’ve seen recently with independant Bundles is Elcodi.

HISTORY

The formal programming concept of objects was introduced in the 1960s in Simula 67, a major revision of Simula I, a programming language designed for discrete event simulation, created by Ole-Johan Dahl and Kristen Nygaard of the Norwegian Computing Center in Oslo.  Simula introduced the notion of classes and instances or objects (as well as subclasses, virtual methods, coroutines, and discrete event simulation) as part of an explicit programming paradigm.

DEFINITIONS. THE BIG 3

Encapsulation is the packing of data and functions into a single component. The features of encapsulation are supported using classes.

Inheritance is when an object or class is based on another object or class, using the same implementation or specifying implementation to maintain the same behavior (realizing an interface; inheriting behavior). It is a mechanism for code reuse and to allow independent extensions of the original software via public classes and interfaces. The relationships of objects or classes through inheritance give rise to a hierarchy.

Polymorphism is when source code dependancy is opposed to the source of control.

DOWNLOADABLES

Symfony2 Routing cheat sheet

Symfony2 Forms cheat sheet