The Quartz64 Model A is powered by a Rockchip RK3566 quad-core ARM Cortex A55 64-Bit Processor with a Mali G-52 GPU. It comes equipped with 2GB, 4GB or 8GB LPDDR4 system memory, and a 128Mb SPI boot flash. There is also an optional eMMC module (up to 128GB) and a microSD slot for booting.
So far I tried the following and worked out without issues: – Installed Balena etcher in Ubuntu, a nice software to make bootable SD’s or eMMC – Downloaded last version of Manjaro-arm for this board (at this moment: 0220606 release) – Connected an external monitor using HDMI, an external wireless logitech Keyboard/Mouse, all recognized automatically by Manjaro
Additionally I wrote a post in Pine64 forum, since I want to know how to go further and test the Eink interface.
This is the result:
Curious to connect a LAN cable afterwards since this does not have WiFi and also trying out a WLAN USB to see if I can also use external WiFi.
lv_port_esp32-epaper is the latest successful attempt to design UX in C using Espressif ESP32. If you like the idea please hit the ★ in my repository fork. What is LVGL?
LVGL stands for “Light and Versatile Graphics Library” and allows you to design an object oriented user interface in supported devices. So far it supports mostly TFT screens and only some slow SPI epapers where supported. My idea is to add driver support so it works also in fast parallel epapers.
The main idea is to use a bridge driver that pushes the pixels to EPDiy component using the set_px_cb and the flush callbacks in order to render the layouts on the supported epapers. This will have a performance hit but it will also allow us to draw UX interfaces in parallel epapers that are quite fast flushing partial refresh. The development took about one month of research and many iterations until it became usable. I started with an easy choice since Lilygo sent me an parallel epaper as a gift once and I bough the rest in their official store. The idea is that this acts as proof-of-concept to demostrate that is possible and that it’s working as expected. It’s possible to design an UX directly in C and then using a controller like ESP32 you can directly interact with Home appliances such as lights or other devices, to control them or to read information such as sensors that can respond with short JSON messages to inform your epaper control board about temperature or other matters that you choose.
After almost one year on the run, my service to deliver images for epapers and TFT displays finally is starting to get some adoption. The idea was starting at the beginning of 2020 when the epapers and many great projects like EPDiy in hackaday started to be early adopted.
Our ESP32 Firmware does 3 things at the moment and is very easy to set up:
It connects to cale.es and downloads a Screen bitmap.
In “Streaming mode” it pushes the pixels to Adafruit GFX buffer and at the end renders it in your Epaper.
It goes to sleep the amount of minutes you define
But then I needed to research more and a bigger idea was triggered: It was not enough to make an Arduino-esp32 firmware using GxEPD as a library. I wanted to learn more how epapers work and also to get out of Arduino-esp32 and get more into Espressif IDF framework. It was hard, I had some weeks where I achieved nothing, but after about one entire month of coding I finally saw the first small epaper refresh. Soon there where 5 models more.
It was a long journey and time taking. But I think it was worth it and I see that at least 20% of the users are having their screens connected and enjoying a very low consumption calendars and photo-frames at home. Very happy to make this possible and to bring something alternative to the usual arduino-esp32 Firmware. Something that you can hack, that is more understandable, and uses Espressif’s own framework. It might be not very well known for makers but is undoubtedly used in professional industry and it’s a very good alternative, with lots of examples and very well documented.
Next missions are to start making developer tools and examples to introduce uGFX interface design into ESP32 using epapers. There is a long journey ahead and we are very thankful for all the good feedback received so far.
Meet Remora-matrix a project that started with the idea to sniff MIDI messages and make simple visuals on a LED-Matrix. Our previous Firmware Remora was intended to receive short commands from ORCΛ and make very simple Neopixels animations on addressable LEDs stripes (WS2812B like) With this one I decided to go a step further and make two versions, one that uses nodejs as a middleware (Requires WiFi and UDP) and other that uses MIDI serial, received directly via Serial2 (TX, RX) UART and hence requires no WiFi connection.
NODEJS VERSION This version uses a middleware script that sniffs a MIDI port (Ex. USB) and converts the messages into UDP messages that fly over WiFi to a destination IP. This script lives at the moment in middleware directory. An npm install needs to be run in order to install the required JS libraries.
// Will list available midi entries. Requires port and udp destination IP
-p, --port_id ID: Port name
0: Midi Through:Midi Through Port-0 14:0
1: USB MIDI Interface:USB MIDI Interface MIDI 1 20:0
-u, --udp_ip [string] [required]
// EX. listen to port 1 USB Midi and forward them to UDP x.x.x.x:49161
// Port is fixed to the ORCA default port, feel free to update it ^
nodejs midi.js -p 1 -u 192.168.12.109
This script will simply run in the background and redirect them using a simple short message that we designed to be compatible with both UDP and Serial versions.
2. MIDI SERIAL version This uses the Sparkfun Arduino midi HAT that cost around 15 u$ and it can be found both in eBay and Aliexpress. The easy task is to build a connecting PCB below that hosts the ESP32 at the side of the HAT with the RX and TX cables from the HAT outputs connected to the ESP32. This HAT has an opto-isolator, also called octocoupler, that converts the MIDI signals into readable UART messages. My prototype construction looks like this:
The MIDI Hat was designed for Arduino and requires 5 volts to run, so the 4 cables wiring is pretty straightforward: HAT midi > ESP32 5v . . . . 5v GND . . GND RX . . . . 26 TX . . . . 27
I’m quite sure the TX goes to TX in the ESP32 but may be also the opposite. There is no standard for this I think. But in case it does not work just invert it, there are only signals, so you won’t break anything for trying this. The advantage of Serial are that have less latence than WiFi. Depending on how clean your connection is, sometimes WiFi UDP packages can become clogged, and get out all together which is quite an undesirable effect if you are working with LIVE music. Also UDP nature is designed to be very fast, but has no order like TCP. It’s possible that a note played comes in different order as expected. Or if WiFi is shared, that while your Router is busy, the packages will accumulate and then sent all together, causing a burst of shapes in the Matrix in a moment that does not correlate with the music. This is not happening with the Serial version since there is no middleware redirecting packages and it cam be run without any PC in the middle. As there are no flying WiFi messages, has less latence and it’s much more reactive and fun to work with. Being the only pitfall being that you need a MIDI cable from your computer or Synthesizer to the MIDI Hat + ESP32 controller. Most LIVE music lighting equipment does not rely on WiFi and there is a good reason for it! Reliability.
Building our own internal Midi messaging system
Since this two versions want to achieve the same goal, that is converting the MIDI played into shapes, I though to create an internal messaging that is shared. Maybe this can be a C++ class or component in the future so it should speak the same language, no matter what version you use. The result is very simple, we will keep Channel internally and will use only Note + Status + Velocity. Status and Channel come in the first byte. Then comes the Note and at the end the Velocity. Once we get the last byte, we will assemble this with the following syntax:
2 chars (HEXA) representing Note played 1 boolean representing Status (1 note on, 0 note off) 2 chars (HEXA) representing VelocityNNSVV Note, Status, Velocity
Example: Playing DO in octave 3 that is 36 in decimal, velocity 60, Note ON message would be:
2413B When the same note is released it could be: 24000
After building the message the channel is analyzed. It can either hear on all channels leaving the constants in platformio.ini to 0. Or hear in 3 different channels (Also 3 instruments) this can be of course modified, but 3 is a good balance, to see something that can be correlated with the music. The configuration for this is on platformio.ini file using build_flags
There you can see that this will only forward packages for channels 1,2,15 all the rest will not be sent to the matrix. There is also an option to ignore Velocity and use a fixed number ( MIDI_FIXED_VELOCITY ) And depending on the song, it could played on a high tone, or in a lower tone. Because our matrix is limited, we need to define BASE_OCTAVE and TOP_OCTAVE so we can have a drawing range. That is the most important midi configuration. It would be desirable to have a “learning phase” where you can simply hear the first 10 seconds of a song and calculate this BASE and TOP margins automatically. This a future idea that might be implemented.
Interpreting the messages
I left just a demo of how to interpret this in C++. As we have 2 different firmware versions, one that listens UDP messages, and another one that get’s MIDI via UART you have to select what to compile editing platformio.ini File:
default_envs = esp32
# Uncomment only one of the folders to select what example to run:
#src_dir = firmware/udp-midi-matrix
src_dir = firmware/midi-in-matrix
Every message at the end triggers a function that draws a shape. And that part is open to every different implementation. For example, you can draw a different shape per channel, like: Ch1 – Usually piano or main instrument – Rectangles Ch2 – Triangles Ch3 – Circles Ch4 – Lines and so on
As said this is just an example, but it’s open to draw anything you want, since we are using GFX over the RGB Led matrix. Also you have the Velocity, that is the pressure that is applied to the key, so you can use this factor to make the shape bigger or change colors. The possibilities are unlimited. There is only one important thing to keep in mind. A note with status 1 should be drawn, but same note with status 0, signalizes that the key was release hence we should delete the shape. At the moment is just an experiment that will may never see the light out of my studio, but nevertheless I wanted to leave this post as a declaration of intentions, in case someone wants to fork this and make his own take.
NeoMatrix let’s you map the RGB Led matrix to use GFX
I writed to Marc Merlin who did the amazing job of adding GFX to FastLED and here I wanted to quote his answer
About FrameBuffer GFX: The good news is that your code will now run mostly unmodified on other displays like LCDs, or RGBPanels, or even display on linux. Like this you can write all your code, run it, and debug it on linux, and then upload it to ESP32 or rPI when it’s done. After that, you can go big!
Codenamed Cal e-pe-de for the initials of E-Paper Display this is basically all what I’ve been doing the last weeks. Learning more about C++ object oriented coding and preparing a class that can be used to send graphic Buffers to different epaper displays. In order to do that I’m documenting on the go and building at the same time CALE-IDF that is our Firmware version for CALE.es but this time built on top of the Espressif IoT Development Framework (IDF)
The mission of this new component is to have a similar library for ESP-IDF that is easier to understand. If possible strictly meeting these requirements:
Object-Oriented Programming with human-readable code and function names
Easy to add a new Epaper display drivers
Easy to implement and send stuff to your Epaper displays
ESP-IDF only (No Arduino classes)
Please find here the Wiki with the models that are already supported in the component:
Leave us a note if you want to try this. We can guarantee that for big epaper displays (>=400 pix. wide) it runs faster than GxEPD also supporting Adafruit GFX fonts and geometric functions. If you tried and it worked out for you please don’t forget to add a ✰ Star and spread the idea.
ESP32-S2 integrates a rich set of peripherals, with 43 programmable GPIOs which can be flexibly configured to provide USB OTG, LCD interface, camera interface, SPI, I2S, UART, ADC, DAC and other common functionality. ESP32-S2 provides the optimal HMI solution for touchscreen and touchpad-based devices. The arduino-espressif32 is getting it’s finals touches of grace before being available and I will just do a small repository and try to test as much as I can and also make some benchmark comparisons.
In the first tests the impression is that most of the arduino framework is already working with a few exceptions. Not all the features are supported yet but they will be soon. Sadly the deepsleep is still not optimized correctly in this first revisions of the Saola board and on 3.4 v I measured 0.79 mA/hour. On a good ESP32 board like tinyPICO it’s 0.08 mA/hour (10 times less) now that’s something you can plug to a battery and make it wake up every hour to do something. Please note this is not a fair comparison at this point. It’s only a test with same firmware, same conditions and bypassing power regulator to see how much it consumes directly powering the board with the 3.3V pin. On the other hand, making processor intensive tests like receiving UDP packets or receiving and decompressing at the same time, it showed similar speed with much less consumption than the ESP32.
But other than this small annoyance in this first board revision it’s a really promising product and very powerful System on a Chip. The feature list is impressive:
Xtensa® single-core 32-bit LX7 microcontroller
43 programmable GPIOs.
Standard peripherals including SPI, I2C, I2S, UART, ADC/DAC and PWM.
LCD (8-bit parallel RGB/8080/6800) interface and also support for 16/24-bit parallel.
Camera interface supports 8 or 16-bit DVP image sensor, with clock frequency of up to 40 MHz.
Please follow the repository and add issues or just leave a comment here if you want me to test any of the features and I will publish the results. Note that the only condition is that the results should be shared with the community on the internet. It’s all open-source, so we will keep the same spirit and share the results with everyone else. Another example of ESP32-S2 in action, controlling 144 RGB Neopixels using Makuna library https://twitter.com/martinfasani/status/1267015931689144320
First of all, this small series of blog posts, assume you have installed and are familiar with Platformio IDE to edit and upload code to your Espressif chips. We ‘ve covered in another post the installation and getting started part with this nice IoT editor. Please refer to this section of Platformio to get the basics right of the editor. Prerequisites: Just grab any ESP32 or ESP8266 and make sure it works. Check that you can see the Serial monitor so we can debug the code.
The platformio.ini project configuration file
This is the configuration file where we can define what are the environment targets (chips) where we want to upload and run our program. For example, we can have the same code, to run in ESP8266 or ESP32 using the same source.
; File: platformio.ini
; This is the default environment but we can also use command line
; or change here to espressif8266 to upload it to this target
default_envs = lolin_d32
platform = https://github.com/platformio/platform-espressif32.git
board = lolin_d32
framework = arduino
monitor_speed = 115200
; set frequency to 80/ 160MHz 160000000L
board_build.f_cpu = 160000000L
platform = espressif8266
board = d1_mini_lite
framework = arduino
monitor_speed = 115200
So this will be our first program, will just include the Arduino framework, start a counter on 0. As every program on Arduino framework, will have a setup() that is executed only once when the chip is powered, and a loop() method that is precisely on a loop. It will just print what you see on setup and then a new line every second. I also invite you to to be curious enough to use the CTRL+LEFT click over Arduino.h and to explore what it does. It includes the framework. Actually it includes the FreeRTOS that is a real-time OS for microcontrollers and also some other important definitions and includes for the Arduino ESP32 framework itself to work. And this is the way to discover any library or thing you include in your program, just CTRL+LEFT and explore the code, trying to understand what function it has. So now let’s code our first little program:
int counter = 0;
At the beginning of the loop() the loop itself would have no sense, since it will add one and go back to 0 all the time.
In the next chapter, we will learn how to use the environments we defined in platformio.ini, to add different codes keeping the same program to run in esp8266 and esp32. Just follow this blog or keep tuned to @martinfasani twitter account to read the next release.
NOTE: Up to here we could also have compiled and run the very same code in an Arduino board and it will do exactly the same. Since we are not using any of the Espressif extra goodies like WiFi, this could be run also in another boards, keep tuned to see what is coming next that will be using WiFi or Bluetooth.
So my first though was Ok nice but this won’t work per se using gxEPD library. Because this great library starts the SPI.begin() withouth parameters hence using ESP32 /ESP8266 default GPIOs for SPI communication. So what I did to test is very simple, I forked GxEPD: https://github.com/martinberlin/GxEPD-config-spi.git
And updated this part of the library to use defines that can be injected using Platformio build_flags:
Note that MISO is already defined in espressif32 Arduino framework.
So now we can inject the other 3 in platformio.ini and get the Waveshare example working:
If we want to test it with CALE we can also use the same defines to reference the EInk wiring in lib/Config/Config.h
// Waveshare ESP32 SPI
int8_t EINK_CS = 15;
int8_t EINK_BUSY = 25;
int8_t EINK_RST = 26;
int8_t EINK_DC = 27;
So it was not a lot of time to get it working with this simple modification. I find it great to inject defines using build_flags. It’s really straight-forward and in fact, if they are only simple defines, all configuration for a project could be done just in platformio.ini
The party breaker
So I was excited to get this working and avoid using this SPI adaptors that go between the ESP32 and the Epaper display…until I got the idea to connect it to my small 3.7 V Lipo that I use to test the ESP32’s with a Diode in between to lower 0.5 v. and found out how much it consumes in deepsleep:
That’s 13 mA per hour. Unless I’m missing something and there is something big that needs to be switched of before deepsleep this EPD driver will eat your battery and your kids for dinner.
So that was the low point of this test. If someone’s got a clue about this high consumption when sleeping please write something in the comments.
UPDATE: In the measurement below I forgot to put the diode in serie, so I was giving 3.8 v to the 3.3 pin. Don’t try it otherwise you may kill the ESP32. Adding the diode it went down to 3.4 v and consumption of course went also down to 10 mA. But is still a lot for deepsleep. In good ESP32 boards like TinyPICO and others, deepsleep consumption is as low as 0.08 mA.
I must say although I’m not proud of all the projects I tried to document in Hackaday I do like a lot udpx and Remora, that are made initially to control Addressable LEDs, but they could be expanded and used for another uses.
This is actually an old project for the agency I work that is finally seeing the light. I started using Waveshare V1 and this is the new 7.5″ e-Paper V2
This will be powered by a TinyPICO ESP32 that claims to consume as little as 20uA when in deepsleep. Let’s see when it’s connected to the Waveshare also in deepsleep ;)
But the idea is to connect, refresh the meeting room labels at each door, and then go to sleep for at least 2 hours. And in wakeup check the day of the week, it it’s Saturday or Sunday when we are not in the office, then just keep on sleeping. There is no need to update stuff when no one will watch.
So designed with low-consumption in mind, will be actually my first real battery project, that I hope last at least 30 days without charging. The Firmware is actually not ultra complicated except the BMP image reading part, that I took mostly from the GxEPD Eink library example, with the add-on that I implemented Zlib compression so download takes one second instead of five. Other than that the only thing that does, is to wake up, query a backend passing an URL that renders that webpage screenshot. The image is sent per SPI and it goes to sleep.
Firmware: 90% done and open source
Case: Work in progress
One of my personal favorites since we are working in team long time already is udpx which is a transmission protocol with decompression support (Zlib and Brotli)
That we are using at the moment to receive animation frames but I can see much more potential in it, like to be used to send small JSON beacons and have real-time charts for anything, they are endless possibilities to use a streaming technology that is proving to work stable. The Max. transport Unit (MTU) of the ESP32 is about 1470 bytes, but you can send much more in that payload if you use Brotli or Zlib, so I see that it can be applied in many other use-cases.
udpx is the first project using Espressif boards where we can use the full connectivity possibilities that the ESP32 offers:
Bluetooth WiFi configuration. That’s one of the main reasons the ESP32 offers Ble and Bluetooth serial, so here we put it to use. udpx-app is an open source Android application that offers Ble/Bluetooth serial configuration to send WiFi credentials and also mDNS discovery using zeroconf
WiFi UDP and decompression. ESP32 has enough memory to pack in one sketch both Brotli, miniz (Zlib) and Bluetooth. I did not believe this initially, but udpx is the proof-of-concept that it can be done.
Pixels is the library that is implemented to do the UDP interpretation, read the headers and send the RGB(w) brigthness values of each LEDs to Neopixels. Initially developed by Samuel our colleague in the USA, we work in team also with Hendrik in Frankfurt am Main to develop a Backend controller that will be able to chain animations and route them to multiple ESP32 controllers every millisecond. So this is a very exciting project, that is going slow because everyone is busy in many things at once, but one that I personally steal hours of sleep every time I can just because it’s interesting and beautiful to work on it.
Last addition was the PIX565 protocol, one that’s been inspired in Spectre’s own take on OctoWiFi LED Controller, but one that has been also used in old game consoles. Didn’t know about this, but Neopixels developer Michael Miller put it in words today:
Games also use alternate bit depth images like 565 and even others; to compress in memory textures. Lots of writeups on that and how shaders can use them to render. The graphics stuff you learn for LEDs often overlaps with gaming techniques.
This streaming little fish is swimming since a while and I’m proud that some music hackers are using it in professional Eurorack setups.
It was initially built for ORCΛ sequencer, and receives short UDP commands of 4 to 6 characters, to launch fast animations at a very high framerate. In the last months it also had the addition that if receives more than 9 bytes, it renders also a Pixel animation frame, so you could send short animations, but also a small animated video.
Remora was also the first product that I could imagine can be a Tindie product, but I must confess that I never selled one, even that I put a price that is mostly the Hardware price. I’m good at building, but not at selling. Everyone has to find their strong point.
“ESP–Mesh reduces the loading of smart light devices on the router by forming a mesh with the smart light device.”
I’ve started months ago getting some smart lights for home and I’ve chosen Osram since it was about half the price than Phillips VUE. But the thing with this systems, though they work nice and with very little configuration, is that you need always a “Gateway”. A central point that receives signals and then send via Radio frequency to the lights what they have to do.
The Espressif ESP-Mesh took a different approach: Every device is a network member making a Mesh of interconnected devices. There is no gateway because there is no need for one.
There is also a very important point that is very interesting if you are a maker like me. It’s open source. That means you don’t need to buy a 60€ light to test it, you can just go to the ESP-Mesh github repository and download ESP-Mesh-Light example to compile it in one of your existing ESP32 Boards. Then you can get easily something like this working :
So what I’m working on the free time to take a rest from another pending projects is to take this PWM output and amplify it using a 74HC125 Quad Bus Buffer to power more than one led.
Command lines to execute the compilation are only 2:
make menuconfig make erase_flash flash
Compile this into your ESP32 and then download Espressif official Android App or if you are using I-phone / I-Pad just search for “ESP-Mesh” The Mesh devices are configured using Bluetooth so keep in mind to have this enabled on your device. Instead of taking the “WiFi Manager” Approach, using an App, you have the benefit that you can just send the WiFi Credentials plus configuration directly to the ESP32.