Installing web screenshot programs in Amazon AWS AMI servers

Installing in my Amazon instance some image manipulation tools, I want to share the two command-lines that saved me a lot of time in this post: (credits given)

Go to https://wkhtmltopdf.org/downloads.html and depending on your version download the CentOS, for me the right was

CentOS 7 x86_64
sudo yum install yourDownloaded.rpm
sudo yum install ImageMagick

Then you can simple take screenshots of the web using :
wkhtmltoimage

and convert them to any format that you like. The image of this post is a screenshot ot the Zeit newspaper converted to -monochrome also:

convert your-image.png -monochrome output-image.png

 

Wemos D1 consumption and VCC pins

So this 6 min boring video was just an experiment to make a small consumption table comparing how the Wemos D1 mini behaves when receiving power through the 5v and 3v3  pins.

5v:
75.5 avg mA only Wifi
126.5 mA with relay on
2.5 mA on deep sleep mode

3v3:
75.4 avg mA only Wifi
119.5 mA with relay on
2.4 mA on deep sleep mode

All code examples will be published in this repository:
https://github.com/martinberlin/esp

Playing with a Wemos D1 (ESP8266)

After learning some Arduino and building some demos I decided to give it a try to another Chips. And this one is the ESP8266, you can find it in eBay or alixpress searching for “wemos D1”

Basically this little guy has kind of the same capabilities of an Atmel chip with less Digital outputs(9) but comes with WiFi built it. So that means you can use it as a station (STA) or a soft access point. That enables to do a lot of stuff, for example deploying this with sensors and being able to read from another device connected with the same Wifi all of them. In turn, you could control the digital outputs, and turn on / off stuff that are connected to the D1.

This is just a small example built in 5 minutes. I just connected a led with a proper resistance to D4. I still do not understand well with pin 2 turns on D4 but that is important now since it’s just a small demo.

UPDATE: Now I understand. You need to use the constants to get the right pins. So using D4 will automatically point it to the right pin that is 2 :) But it’s just much simpler to use the constants unless you want to go all the time to the datasheet. After building this simple example I build a bigger one that can send a message from the web to a simple display and show some weather measurements. Will share it later on a new post if there is any interest about it. You can comment here if it would be interesting for you to read it.

NOTE: I made a mistake here and connected the 3.7 V. to the 3.3V pin. That is incorrect! It should be connected to the 5V pin that in turn goes to the power regulator
/*
* Based on ESP8266 LLMNR responder sample
* Copyright (C) 2017 Stephen Warren 
*
* Based on:
* ESP8266 Multicast DNS (port of CC3000 Multicast DNS library)
* Version 1.1
* Copyright (c) 2013 Tony DiCola (tony@tonydicola.com)
* ESP8266 port (c) 2015 Ivan Grokhotkov (ivan@esp8266.com)

* Instructions:
* - Update WiFi SSID and password as necessary.
* - Flash the sketch to the ESP8266 board.
* - Point your browser to http://ip-you-get-in-serial-monitor-arduino/
*
*/

#include 
#include 
#include 
#include 

const char* ssid = "yourRouterSSID";
const char* password = "yourPassword";

ESP8266WebServer web_server(80);

void handle_http_not_found() {
  web_server.send(404, "text/plain", "Not Found");
}

void handle_http_root() {
  // NOTE: Including bootstrap here just to make fancy buttons
  String headers = "	";
  String html = "Server options:
";
  html += "<a class='btn btn-primary' href='4/1' target='frame'>D4 on</a>
";
  html += "<a class='btn btn-primary' href='4/0' target='frame'>D4 off</a>
";
  html += "";
  web_server.send(200, "text/html", headers + html);
}

void handle4off() {
digitalWrite(2, 0);
web_server.send(200, "text/html");
}
void handle4on() {
digitalWrite(2, 1);
web_server.send(200, "text/html");
}

void setup(void) {
Serial.begin(115200);
pinMode(2, OUTPUT);
digitalWrite(2, 0);

// Connect to WiFi network
WiFi.mode(WIFI_STA);
WiFi.begin(ssid, password);
Serial.println("");

// Wait for connection
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("");
Serial.print("Connected to ");
Serial.println(ssid);
Serial.print("IP address: ");
Serial.println(WiFi.localIP());

// Start LLMNR responder
LLMNR.begin("esp8266");
Serial.println("LLMNR responder started");
// Start HTTP server
web_server.onNotFound(handle_http_not_found);
web_server.on("/", handle_http_root);
web_server.on("/4/1", handle4on);
web_server.on("/4/0", handle4off);
web_server.begin();

Serial.println("HTTP server started");
}

void loop(void) {
web_server.handleClient();
}

So after uploading this little script to your Wemos and of course updating your SSID and password to your router it should act like a server. Notice that I’m using a hidden Iframe to keep things simple, since I see other examples that submit a Form to itself and analyze the arguments. That is cool maybe for a bigger applicaion, but for simple apps, is enough to just point to an URL and reply with a 200 Http status.

Penta-light uploaded to Thingiverse

El pase de diapositivas requiere JavaScript.

Designed in Blender just for fun, it’s a small 5 Led-diode lamp, that is powered by 3 AAA batteries. What I like about pentagons is that it does not matter from what angle you see the iluminated object, there are always 3 lights to see.

With the 100 Ohm resistances the light should stay about 30 hours long (depending of what rechargable batteries are used)

If you want more details the object can be downloaded here:
https://www.thingiverse.com/thing:2826520

 

Starting again with Arduino and 3D design

About a month ago I joined the amazing community of Thingiverse and started uploading my first designs made on Blender.

One of the first projects is to control automatically the curtain blinds to ensure we don’t have to get up to turn them manually. Actually it’s just a great excuse to learn more about arduino and stepper motor controlling.

This is the basic schema to control light detection:

Then I just connected some leds to the digital output:


int ledPinMin = 2;
void setup() {
Serial.begin(9600);
pinMode(ledPinMin, OUTPUT); // declare the ledPin as an OUTPUT
pinMode(ledPinMin+1, OUTPUT);
pinMode(ledPinMin+2, OUTPUT);
pinMode(ledPinMin+3, OUTPUT);
pinMode(ledPinMin+4, OUTPUT);
pinMode(ledPinMin+5, OUTPUT);
}
void loop() {
int sensorValue = analogRead(A5);
int ledPin = sensorValue/50 +2;
digitalWrite(ledPin, 1);
delay(sensorValue);
digitalWrite(ledPin, LOW);
delay(100);
}

I really like the projects like arduino and the power that brings 3D printing to the table. It’s a great combination. Anything can be done you just need to have an idea and be creative enough to make it happen.

How routing internally works in Symfony 4 framework

At the moment of writing this I’m testing the Symfony 4 Framework after reading the following article and also Fabian’s latest posts on Medium.  I’m testing weather the not yet officially released version, will be a future candidate for some new project or not. But the interesting part of this research is that I already realized what I won’t do in new projects. But let’s focus first on the mission of describing how routing works internally in the framework.
So we will go from the Frontend Controller up to the internal guts that say from the route /test let’s execute Controller:test()

http:// bestpractices.local/test

The front-end controller located in /web/index.php bootstraps the framework:

$kernel = new Kernel(getenv('APP_ENV'), getenv('APP_DEBUG'));
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();

As we already know all this framework is based on Request -> Response concept from the beginning to the latest controller action and the Kernel is no exception.  If we open this Kernel we can see that it’s mission is to setup Cache and Logs directories, register the bundles, configure the Container and Routing.

To go deeper in the chain we need to open HttpKernel with the mission described as:
HttpKernel notifies events to convert a Request object to a Response one.
There on the handleRaw method “Handles a request to convert it to a response” is where the whole framework workflow takes place:

namespace Symfony\Component\HttpKernel;

class HttpKernel implements HttpKernelInterface, TerminableInterface {
private function handleRaw(Request $request, $type = self::MASTER_REQUEST)
    {
        $this->requestStack->push($request);
        // request
        $event = new GetResponseEvent($this, $request, $type);
        $this->dispatcher->dispatch(KernelEvents::REQUEST, $event);

  // At this point the routing is already resolved
  // dump($request->attributes);exit();

        if ($event->hasResponse()) {
            return $this->filterResponse($event->getResponse(), $request, $type);
        }

        // load controller
        if (false === $controller = $this->resolver->getController($request)) {
            throw new NotFoundHttpException(sprintf('Unable to find the controller for path "%s". The route is wrongly configured.', $request->getPathInfo()));
        }

// ... Return a response

As we can see on the first lines, the Routing part, is resolved dispatching an Event called KernelEvents::REQUEST

namespace Symfony\Component\HttpKernel\EventListener;

class RouterListener implements EventSubscriberInterface {

public function onKernelRequest(GetResponseEvent $event)
    {
        $request = $event->getRequest();

        $this->setCurrentRequest($request);

        if ($request->attributes->has('_controller')) {
            // routing is already done
            return;
        }

        // add attributes based on the request (routing)
        try {
            // matching a request is more powerful than matching a URL path + context, so try that first
            if ($this->matcher instanceof RequestMatcherInterface) {
                $parameters = $this->matcher->matchRequest($request);
                // dump($parameters); // here to see it matched

            } else {
                $parameters = $this->matcher->match($request->getPathInfo());

            }

And in this part of the code we can see that dumping the $parameters the Request attributes are already assigned

$parameters = array(
  "_controller" => "App\Controller\DefaultController::testAction"
  "_route" => "test"
);

Note: Request->attributes is used to store information about the current Request such as the matched route, the controller, etc
But more interesting is to replace this dump for a : dump($this->matcher);
Then we can see that the real matcher of the routing is done by srcDevDebugProjectContainerUrlMatcher since we are currently using DEV as an environment.

This file, that we can find in this path /var/cache/dev/srcDevDebugProjectContainerUrlMatcher.php , is the responsible of returning the request parameters with the right controller to be executed for this route.
Since doing all the routing on the fly will be a real show stopper and we are concerned with performance: reading the configuration from the filesystem may slow down the application.

That’s why there’s a PhpMatcherDumper class which can generate an implementation of UrlMatcherInterface with all configuration in an optimized way.

Even in DEV, this UrlMatcher file, is only rendered again if we update any part of the routing. That’s one of the reasons clearing all cache will make a slow first request.

Lessons learnt from this research are:

  • Keep your routes clean
    Not used / deprecated routes that are still there will make this UrlMatcher unnecessary long and hence slower to resolve.
  • Avoid at all costs using Annotations for the routing
    It’s generally considered a bad practice in OOP and contains much more Cons than Pros
  • Avoid using Assetic
    This generates fully unnecessary routes for every assset in your HTML templates. New versions of Symfony will not come with Assetic so it’s better to get along without it.

I never care to check this routing to the core before and I highly recommend from time to time to check the internals of Symfony. You will learn a lot of the framework essentials and also will be more cautious of how to take the best of it. In my personal case, I learnt much more doing this small research taking your own conclusions, than reading some other developer top 5 list on framework optimization.

The code to reproduce this can be downloaded from public repository:
https://github.com/martinberlin/symfony-flex

Catalunya declared their independence today 27 October 2017

catalunya

About 15.30 the Parlament ha aproved the new Catalán Republic with 70 YES votes and 10 NO votes. What happens next no one knows.

Since I lived in Mallorca for about 12 years I always dreamed to come back to Spain next to where my fathers live in Catalunya, near Arenys de Mar. So now I’m quite worried about what is going to happen next. My wish is that whatever happens next is without violence and represion like in 1-O vote. Violence can never be the answer to anything.

Integrating SSO in a project using Shibboleth authentication

Shibboleth is an open-source identity federation solution that’s used in research and education communities worldwide.

Graphic by ncsu.edu

Basically is a federating agent that consists of 2 components:

1- Service Provider (SP)
 which request attributes, represented in this implementation als storage.luckycloud.de using Apache2-mod-shib 
2- Identity Provider(IDP)
 which broadcasts attributes, and will be implemented in the Login-logic of the site

Now the SP is actually out-of-the-box with apache2 module mod-shib. But my interest here is to do the Identity Provider part als Symfony module.

 

Is someone else out there interested on following the implementation ?