Hackaday projects update: CALE Eink Calendar and udpx

I must say although I’m not proud of all the projects I tried to document in Hackaday I do like a lot udpx and Remora, that are made initially to control Addressable LEDs, but they could be expanded and used for another uses.


This is actually an old project for the agency I work that is finally seeing the light. I started using Waveshare V1 and this is the new 7.5″ e-Paper V2

CALE E-eink calendar ESP32
CALE Eink calendar ESP32

This will be powered by a TinyPICO ESP32 that claims to consume as little as 20uA when in deepsleep. Let’s see when it’s connected to the Waveshare also in deepsleep ;)
But the idea is to connect, refresh the meeting room labels at each door, and then go to sleep for at least 2 hours. And in wakeup check the day of the week, it it’s Saturday or Sunday when we are not in the office, then just keep on sleeping. There is no need to update stuff when no one will watch.
So designed with low-consumption in mind, will be actually my first real battery project, that I hope last at least 30 days without charging. The Firmware is actually not ultra complicated except the BMP image reading part, that I took mostly from the GxEPD Eink library example, with the add-on that I implemented Zlib compression so download takes one second instead of five. Other than that the only thing that does, is to wake up, query a backend passing an URL that renders that webpage screenshot. The image is sent per SPI and it goes to sleep.
Firmware: 90% done and open source
Case: Work in progress
CLIENT: http://www.skygate.de


udpx sending frames to a 44*11 RGB Led matrix
One of my personal favorites since we are working in team long time already is udpx which is a transmission protocol with decompression support (Zlib and Brotli)
That we are using at the moment to receive animation frames but I can see much more potential in it, like to be used to send small JSON beacons and have real-time charts for anything, they are endless possibilities to use a streaming technology that is proving to work stable. The Max. transport Unit (MTU) of the ESP32 is about 1470 bytes, but you can send much more in that payload if you use Brotli or Zlib, so I see that it can be applied in many other use-cases.

udpx is the first project using Espressif boards where we can use the full connectivity possibilities that the ESP32 offers:

  1. Bluetooth WiFi configuration. That’s one of the main reasons the ESP32 offers Ble and Bluetooth serial, so here we put it to use.
    udpx-app is an open source Android application that offers Ble/Bluetooth serial configuration to send WiFi credentials and also mDNS discovery using zeroconf
  2. WiFi UDP and decompression. ESP32 has enough memory to pack in one sketch both Brotli, miniz (Zlib) and Bluetooth. I did not believe this initially, but udpx is the proof-of-concept that it can be done.

Pixels is the library that is implemented to do the UDP interpretation, read the headers and send the RGB(w) brigthness values of each LEDs to Neopixels. Initially developed by Samuel our colleague in the USA, we work in team also with Hendrik in Frankfurt am Main to develop a Backend controller that will be able to chain animations and route them to multiple ESP32 controllers every millisecond. So this is a very exciting project, that is going slow because everyone is busy in many things at once, but one that I personally steal hours of sleep every time I can just because it’s interesting and beautiful to work on it.
Last addition was the PIX565 protocol, one that’s been inspired in Spectre’s own take on OctoWiFi LED Controller, but one that has been also used in old game consoles. Didn’t know about this, but Neopixels developer Michael Miller put it in words today:

Games also use alternate bit depth images like 565 and even others; to compress in memory textures. Lots of writeups on that and how shaders can use them to render. The graphics stuff you learn for LEDs often overlaps with gaming techniques.


This streaming little fish is swimming since a while and I’m proud that some music hackers are using it in professional Eurorack setups.

It was initially built for ORCΛ sequencer, and receives short UDP commands of 4 to 6 characters, to launch fast animations at a very high framerate. In the last months it also had the addition that if receives more than 9 bytes, it renders also a Pixel animation frame, so you could send short animations, but also a small animated video.

Remora was also the first product that I could imagine can be a Tindie product, but I must confess that I never selled one, even that I put a price that is mostly the Hardware price. I’m good at building, but not at selling. Everyone has to find their strong point.


PHP Symfony versioning and composer

Lately I’ve been switching to new versions and upgrading Symfony for client projects that I cannot reveal, but what I would like to share in this simple and short post is what combinations worked for me after many hours switching versions and dependancies.

Symfony 4

    "require": {
        "php": "^7.1.3",
        "ext-ctype": "*",
        "ext-iconv": "*",
        "sensio/framework-extra-bundle": "^5.4",
        "symfony/asset": "4.3.*",
        "symfony/console": "4.3.*",
        "symfony/dotenv": "4.3.*",
        "symfony/expression-language": "4.3.*",
        "symfony/flex": "^1.3.1",
        "symfony/form": "4.3.*",
        "symfony/framework-bundle": "4.3.*",
        "symfony/http-client": "4.3.*",
        "symfony/intl": "4.3.*",
        "symfony/monolog-bundle": "^3.1",
        "symfony/orm-pack": "*",
        "symfony/process": "4.3.*",
        "symfony/security-bundle": "4.3.*",
        "symfony/serializer-pack": "*",
        "symfony/swiftmailer-bundle": "^3.2",
        "symfony/translation": "4.3.*",
        "symfony/twig-bundle": "4.3.*",
        "symfony/validator": "4.3.*",
        "symfony/web-link": "4.3.*",
        "symfony/webpack-encore-bundle": "^1.6",
        "symfony/yaml": "4.3.*",
        "jms/serializer-bundle": "3.4.1",
        "friendsofsymfony/user-bundle": "2.1.2"

This is the easiest one since is the last version and you can just install it following the docs. Added some nice packages that I use always just to JSON serialize entities.

Symfony 3.4

"require" : {
        "php" : "^7.1",
        "symfony/symfony" : "3.4.x-dev",
        "symfony/webpack-encore-bundle" : "~1.6",
        "symfony/swiftmailer-bundle" : "3.3.*",
        "symfony/monolog-bundle" : "3.4.0",
        "symfony/console" : "^3.4",
        "symfony/serializer": "^3.4",
        "sensio/distribution-bundle" : "5.0.25",
        "sensio/framework-extra-bundle" : "5.4.0",
        "doctrine/orm": "2.6.4",
        "doctrine/doctrine-bundle": "2.0.x-dev",
        "twig/extensions" : "1.5.4"

Basic install. This where you would like to go in case you are still using 2.8 version that will be soon out of maintenance

Symfony 3.3

"require": {
        "php": ">=5.6.27",
        "symfony/symfony": "^3.3",
        "doctrine/orm": "^2",
        "doctrine/doctrine-bundle": "1.10.0",
        "symfony/swiftmailer-bundle": "~2",
        "symfony/monolog-bundle": "^3",
        "sensio/distribution-bundle": "~5.0",
        "sensio/framework-extra-bundle": "^3",
        "incenteev/composer-parameter-handler": "~2.0",
        "knplabs/knp-snappy-bundle": "~1.4",
        "knplabs/doctrine-behaviors": "~1.4",
        "friendsofsymfony/user-bundle": "~2.0@dev",
        "doctrine/doctrine-fixtures-bundle": "^2.3",
        "twig/extensions": "^1.4",
        "mopa/bootstrap-bundle": "dev-master",
        "m6web/guzzle-http-bundle": "~2.0",
        "doctrine/migrations": "^1.5",
        "doctrine/doctrine-migrations-bundle": "^1.3",
        "jms/serializer-bundle": "1.5.0"

The important difference in this 3.3 version is that it still runs on PHP 5.6+ but if you can it would be the best to go to 3.4 or even 4 but that requires a big refatoring in many corners.

Cordova cross platform mobile applications

After the small Chrome App experience that resulted in a very compact app that reads Video and sends an UDP binary stream to ESP32 Led controllers I decided to start learning how to program a real mobile app.
Real because I find that the way to do it must end in the App Store and the user should be able to install it with a few clicks.
This start guide is only valid for Linux but it could be a similar approach in Mac.

1.- Install Java 8 SDK

It’s very straight-forward to start if you have previous html/javascript experience building a web page.

In this github repository:

I’m building my very first Android app. Follow the commits and watch the Repository to see my new steps and also my failures in this new world.

This is a preview of how the app looks like in it’s first version:

udpx Android application is available in the Play store.
udpx app

Soon more posts will follow explaining how to interface with Android OS (Camera as an example). There will be soon more issues to expand and implement in this repository. And my new philosophy to build Firmware for Espressif chips will be simply:

“One firmware, one mobile application”

Meaning that every firmware will have his app, where you can configure and test it, just with one click install in google Play store and setting up some very easy configuration. Starting from WiFi credentials using BLE (Bluetooth low energy) and setting up the App IP address as a last step.

My goal at last, is that internet of the things is easy to use, and not something where you need a 20 minutes complicated setup to start using something. So the mission is that for first weeks 2020 all master branches from our main ESP32 Firmwares like udpx and Remora should be bluetooth configurable.

Building Visual Code in a Raspberry Pi

When I’m on the road without laptop I miss having something to compile Espressif boards. So I decided to document this a little bit.
Visual Code is the base for Platformio that is my IDE of choice when coding in C / C++

It turns out that for ARM like many other software you need to go the hard way to build it since there is no Debian package ready for this architecture. So to start with, you need to have latest version of Node for this arm version: uname -m
will tell you what version of ARM you are on. Then just follow Node instructions to get the latest version on your system. After node installation, there are some extra packages that are needed for the build:

sudo apt install libx11-dev libxkbfile-dev libsecret-1-dev

Then just follow this guide and get some cofee because it takes at least 20 minutes to build:
Building Visual Studio Code on a Raspberry Pi 3
Official repository: https://github.com/microsoft/vscode

I must say I’m really impressed at the stuff you can build using npm/nodejs this times. Combined with cross platform desktop apps like Electron it’s really amazing to see how the community is building software with this.

Using the Extension marketplace

It’s important to note that the build version of VSCODE does not support the official marketplace. So in order to install Platformio you will need to use the extended … menu in Extensions and use “Install from VSIX” and first install C/C++ :

1. https://github.com/Microsoft/vscode-cpptools/releases Select cpptools-linux.vsix
2. And then search for Platformio VSIX
Install them in this order since Platformio requires cpptools otherwise it won’t start. It’s a pity that Microsoft did not enable the Marketplace for VSCODE builds but at least there is a way to install extensions manually.

Some very useful out-topic commands in Debian/Raspbian

// List installed packages
sudo dpkg-query

// Create a list of all installed packages
sudo dpkg-query -f '${binary:Package}\n' -W > packages_list.txt

// Using apt
sudo apt list --installed | less

Met Remora a simple firmware and language to launch addressable led animations from ORCΛ

Remora listens to UDP short commands to trigger LED Animations. Receives ; commands from ORCΛ

As a side fun idea while learning Nodejs and while working for a bigger and more complex project with my partner Hendrik I programmed Remora just to have an easy way to launch animations from ORCΛ

If you want to try it out please check the requirements and components on the repository or on the hackaday project page.
After getting addressable LEDs and any ESP32 board, it’s easy to compile this, reading the provided instructions with Arduino or Plataformio. It needs simple configuration, to know your WiFi credentials and the length of your led stripe.
After initializing and getting online, the firmware will listen by default in port 49161 and launch short animations as fast as the UDP message hits the chip. To test it send an UDP message using netcat or any other way. In Linux that can be done like this:

echo '82O'|nc -w1 -u ESP32_IP 49161

IMPORTANT: If you copy the upper text make sure that the quotes are single quotes otherwise the compiled C will not recognize what animation to start

That is enough if you want to send animations to one controller. If you need more then we created a simple UDP redirector script that lives in the extras folder. This can be run using:

nodejs udproxy.js

After configuring a simple ip-config.json file that has an id to IP lookup table:


When this is running in the background, it will just process any message coming, extract the first character and pick up that ID from the table. Then it will redirect the rest of the message to that IP, same port 49161. To explain it more graphically, sending:

echo '282O'|nc -w1 -u localhost 49161

will lookup IP 2 and send 82O to enabling you to send animations to different LED Stripes from Orca sequencer software.

Remora Logo


Electronic schema

Additional info

Public repository
https://github.com/martinberlin/Remora Stable branch: master (only ESP32)
Hackaday project page for Remora – Find component list there.
Approximate cost of hardware not including 3DPrint: 19€ or about 22 dollars

UDP + Brotli compressed messages on Microcontrollers: Sending more with less

I wanted to document on this paper my efforts on researching a way to send efficiently a lot of data to an ESP32.

One of the attractive features of UDP is that since it does not need to retransmit lost packets nor does it do any connection setup, sending data incurs less delay.

Wikibooks on Communications Networks UDP Protocol

That made it to be very good choice to transmit audio or video over the Web. But in my case-study, working for a LED stripe hardware controller in team with @hputzek, we where interested in high framerate transmission with the less possible bandwidth use.
I first researched in the direction of zlib (miniz) that is already in the ESP32 Core in it’s minimal expression. I could not get it to work since I did not know back then how to convert incoming bytes into their corresponding integer. But here is a solution if someones wants to try the miniz zlib compression/decompression out. It’s already on ESP32 Rom, so I guess it should have less overhead than any other decompression method.

Then looking forward other decompression methods Brotli come in the scene:

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a great compression ratio

Brotli library definition

The goal was to send a byte for each Red, Green and Blue values of a 144 LEDS Stripe but with the same technology would be possible to send fast samples of complex sensor metrics.
Test samples are here available. Samples of 1.45 KB bytes where compressed with zlib (85 bytes) and with Brotli (57 Bytes) giving a clear win over Brotli compression. Speed of decompression was between 1 and 2 ms on the microprocessor.
Here is the udp-receive (server) test that I made just as a raw proof-of-concept:

Compiling that on a ESP32 it’s possible to test the uncompressed result via Serial sending with any tool (Like netcat) a brotli compressed bytestream to the ESP32 IP:

cat 144.txt.br |nc -w1 -u ESP_IP_ADDRESS 1234

Check also the tests/udp-receive branch for a more advanced example using xTaskCreatePinnedToCore. Seeing this work made us choice Brotli as a decompressor and UDP as transport protocol. Thanks to @bitluni for your suggestion last weekend.

This a sample of the result of reading a fast UDP stream of compressed bytes and sending the output to an addressable Led stripe.

Exploring ESP32 Mesh technology

On last short holidays on Barcelona I went to visit my father and I brought with me a very small present:

Well actually the second one is for him, but the other that was made for my brother is still there, so we decided to run some tests. One of the cool things I find about ESP-IDF as a framework instead of Arduino framework for Espressif chips is that before compiling you can do a : make menuconfig

So basically you can set the configuration variables before compiling. Not only from main code but also for any component that is loaded using git submodules. This is how the Light example configuration looks like:

As you can see the RGB Gpios are fully configurable and also the color A to color B transition

So that’s the benefit of building your own light, you can just tweak it, to make a slow transition from Red to Green or add any submodule (Like this ESP32 Mesh Light Oled tweak I published in github)

So what my father did after I installed him the ESP-IDF / MDF stuff in his Ubuntu laptop is to debug the Lamp logs to see how they behave, to analyze the Root-> Nodes behaviour as described in the Mesh API Guide.

Actually only ONE Lamp acts as a root node. All other lamps are getting the instructions from this root node

If the root node lamp is disconnected, after no receiving beacon frames that is kind of a heartbeat to prove root node is alive, child Lamps organize a voting session to see who is going to become the next root node. We can see this clearly in this Log:

FIRST LAMP CONNECTS [0;32mI (695) phy: phy_version: 4008, c9ae59f, Jan 25 2019, 16:54:06, 1, 0 [0m 14:40:26.181 -> I (696) wifi: mode : sta (3c:71:bf:a9:45:14) 14:40:26.181 [0;32mI (745) [mwifi, 138]: esp-mdf version: 67a0a5b [0m 14:40:26.181 -> W (745) wifi:[beacon]new interval:100ms 14:40:26.181 -> I (747) wifi: mode : sta (3c:71:bf:a9:45:14) + softAP (3c:71:bf:a9:45:15) 14:40:26.185 -> I (751) wifi: Init max length of beacon: 752/752 MESH IS STARTED 14:40:27.204 -> I (1773) mesh: ;need_scan:0x1, need_scan_router:0x0, look_for_nwk_count:1 14:40:27.204 -> [0;32mI (1774) [mwifi, 98]: MESH is started [0m 14:40:27.204 -> [0;32mI (1778) [light, 655]: event_loop_cb, event: 0x0 [0m 14:40:27.204 -> [0;32mI (1783) [light, 660]: MESH is started [0m I (2075) mesh: find root:ESPM_D9C830, root_cap:2(max:256), new channel:1, old channel:0 14:40:27.510 -> I (2075) mesh: [S2]MiFibra-509A, 4c:1b:86:5a:50:9c, channel:1, rssi:-28 14:40:27.510 -> I (2078) mesh: find router:[ssid_len:12]MiFibra-509A, rssi:-28, 4c:1b:86:5a:50:9c(encrypted), new channel:1, old channel:0 SECOND LAMP IS CONNECTED 14:40:27.544 -> [0;32mI (2125) [light, 683]: the root connects to another router with the same SSID [0m I (2428) mesh: [SCAN][ch:1]AP:6, other(ID:0, RD:0), MAP:1, idle:0, candidate:1, root:1, topMAP:0[c:0,i:0][4c:1b:86:5a:50:9c]router found 14:40:27.850 -> I (2431) mesh: 6545[selection]try rssi_threshold:-78, backoff times:0, max:5 14:40:27.884 -> I (2438) mesh: [DONE]connect to parent:ESPM_D9C830 (This is the hidden SSID from ROOT Node Lamp), channel:1, rssi:-41, 30:ae:a4:d9:c8:31[layer:1, assoc:0], my_vote_num:0/voter_num:0, rc[00:00:00:00:00:00/-120/0] D (3513) [light, 168]: Erase restart count [0m I (3961) wifi: n:1 1, o:1 0, ap:1 1, sta:1 1, prof:1 14:40:29.375 -> I (3963) wifi: state: init -> auth (b0) I (3971) wifi: state: auth -> assoc (0) 14:40:29.409 -> I (3979) wifi: state: assoc -> run (10) 14:40:29.409 -> I (3980) wifi: connected with ESPM_D9C830, channel 1 NOTE: I guess restart count is a counter to reset the Lamp configuration, since turning it on/off for 3 times, resets the Lamp and blinks yellow waiting for configuration. SO NOW, WE DISCONNECT ROOT NODE (First Lamp) The node child has no parent, hence no connectivity at all and does not receive any commands, this is what happens [light, 140]: System information, channel: 1, layer: 2, self mac: 3c:71:bf:a9:45:14, parent bssid: 30:ae:a4:d9:c8:31, parent rssi: -34, node num: 2, free heap: 159120
[light, 655]: event_loop_cb, event: 0x8 [0m 14:44:22.587 -> I (237153) mesh: [wifi]disconnected reason:200(beacon timeout), continuous:1/max:12, non-root, vote(,stopped) 14:44:22.587 -> [0;32mI (237158)
[light, 669]: Parent is disconnected on station interface [0m 14:44:22.620 -> I (237168) mesh: [scan]new scanning time:600ms And after this starts the voting to see what Lamp is going to be the next Root lamp.

The interesting part of a Mesh network is that is like a self-healing network, where the hierarchy is redefined once you remove one of it’s pieces, and reconstructs automatically. Of course this takes time, and if you remove a Root node, all your system will be idle for some seconds (in our tests was up to 20/30 seconds) till the selection of a new Root node takes place. But the benefit, is that as every lamp can be connected to it’s parent and at the same time be an Access point where childs connect, you can extend your WiFi reach. Every lamp is like a small WiFi that broadcasts commands to it’s childs.

Espressif ESP-Mesh updates their App with a new UX

About a month ago I started learning to use ESP-IDF and got very interested in Espressif Mesh Lamps. It’s a lot of fun for me to create my own ESP-Mesh LEDs lamps with this technology. But as with everything I touch that is open-source, apart of being a user, I try to collaborate and made the thing better. That’s the spirit of open source, you are a user, but also at the same time your opinion counts and usually it’s welcomed.

Comparison from previous ESP-Mesh App and new upcoming version:

2018 version
Esp-Mesh App
03/2019 new version

Updates resume

  • More integrated and compact UX. Now On/Off switch is in the same screen allowing for easier usability
  • Doing White with RGB is now possible. Click on the center and the light will turn on the 3 colors at the same time (New feature)
  • Warm/Cold switch is also better signalized and more usable.

I really like the update and I think is a significant improvement over the last version.

Here some selfish pictures of my last lamp projects ;)

Esp-Mesh Lamp with modified software to enable Oled-display (Heltec WiFI32 board)
Esp-Mesh with one RGB 8Watts LED
Esp-Mesh mini projector (Stage type traditional lamp)

ESP-IDF Libraries in Arduino framework

I started to dig a bit more into the Iot development network from Espressif with the intention to slowly learn something new. All the official examples of Mesh lights, ESP32-Camera and ESP-WHO that presents a preliminary version of face recognition are built on the top of the IDF.

So sooner or later, I think will be the official way to do “internet of the things” devices with these Boards. But I’m still not ready for it, I need to do simple examples and experimenting before taking over and do something for real.

So the first things I tackled on is to take this libraries “as is” and use them in existing Arduino framework projects. And it’s fully possible, below is a small example about this using the TTGO Camera with Pir sensor


So far it features:

  • camera_index.html template loaded from the SPIFFS so you can modify it for your project (No Gziped non-editable file)
  • Added V-Flip setting that was not existing in the original example
  • Libraries loaded using platform.io file

This is my humble try to make a hackeable and modificable esp32 Camera example using ARDUINO as a framework but with the official Espressif Camera Libraries.

One important step is that without using the IDF, just adding the libraries in plataform.io project configuration file:

platform = espressif32
framework = arduino

lib_deps =

It won’t compile. The reason is that it tries to find the C header files but they do not have the needed mapping out of the box if you use framework: arduino. It will complain that:

Compiling .pioenvs/ttgo-lora32-v1/libe0c/esp32-camera/conversions/jpge.cpp.o
.piolibdeps/esp32-camera/conversions/jpge.cpp:11:18: fatal error: jpge.h: No such file or directory

And the file is there but in another directory. So my workaround was to add this directory in the build_flags configuration:

build_flags =

After this it compiles without errors.

Some notes and references for this post

Think about C header files as your interfaces, your function prototypes! Code organization is very important.
3D-models for the TTGO Cameras available in Thingiverse, my own prototype design (be aware!)

New ESP-Mesh RGB+White Light soldered. You can do your own for about 10€ materials and 2 hours soldiering ;)
Small preview:

Starting to do smart-home appliances with ESP-Mesh

ESPMesh reduces the loading of smart light devices on the router by forming a mesh with the smart light device.”

I’ve started months ago getting some smart lights for home and I’ve chosen Osram since it was about half the price than Phillips VUE. But the thing with this systems, though they work nice and with very little configuration, is that you need always a “Gateway”. A central point that receives signals and then send via Radio frequency to the lights what they have to do.

The Espressif ESP-Mesh took a different approach: Every device is a network member making a Mesh of interconnected devices. There is no gateway because there is no need for one.

There is also a very important point that is very interesting if you are a maker like me. It’s open source. That means you don’t need to buy a 60€ light to test it, you can just go to the ESP-Mesh github repository and download ESP-Mesh-Light example to compile it in one of your existing ESP32 Boards. Then you can get easily something like this working :

So what I’m working on the free time to take a rest from another pending projects is to take this PWM output and amplify it using a 74HC125 Quad Bus Buffer to power more than one led.

For those interested in reproducing this example:


Command lines to execute the compilation are only 2:

make menuconfig
make erase_flash flash

Compile this into your ESP32 and then download Espressif official Android App or if you are using I-phone / I-Pad just search for “ESP-Mesh”
The Mesh devices are configured using Bluetooth so keep in mind to have this enabled on your device. Instead of taking the “WiFi Manager” Approach, using an App, you have the benefit that you can just send the WiFi Credentials plus configuration directly to the ESP32.