After the small Chrome App experience that resulted in a very compact app that reads Video and sends an UDP binary stream to ESP32 Led controllers I decided to start learning how to program a real mobile app.
Real because I find that the way to do it must end in the App Store and the user should be able to install it with a few clicks.
This start guide is only valid for Linux but it could be a similar approach in Mac.
Soon more posts will follow explaining how to interface with Android OS (Camera as an example). There will be soon more issues to expand and implement in this repository. And my new philosophy to build Firmware for Espressif chips will be simply:
“One firmware, one mobile application”
Meaning that every firmware will have his app, where you can configure and test it, just with one click install in google Play store and setting up some very easy configuration. Starting from WiFi credentials using BLE (Bluetooth low energy) and setting up the App IP address as a last step.
My goal at last, is that internet of the things is easy to use, and not something where you need a 20 minutes complicated setup to start using something. So the mission is that for first weeks 2020 all master branches from our main ESP32 Firmwares like udpx and Remora should be bluetooth configurable.
When I’m on the road without laptop I miss having something to compile Espressif boards. So I decided to document this a little bit. Visual Code is the base for Platformio that is my IDE of choice when coding in C / C++
It turns out that for ARM like many other software you need to go the hard way to build it since there is no Debian package ready for this architecture. So to start with, you need to have latest version of Node for this arm version: uname -m will tell you what version of ARM you are on. Then just follow Node instructions to get the latest version on your system. After node installation, there are some extra packages that are needed for the build:
I must say I’m really impressed at the stuff you can build using npm/nodejs this times. Combined with cross platform desktop apps like Electron it’s really amazing to see how the community is building software with this.
Using the Extension marketplace
It’s important to note that the build version of VSCODE does not support the official marketplace. So in order to install Platformio you will need to use the extended … menu in Extensions and use “Install from VSIX” and first install C/C++ :
Remora listens to UDP short commands to trigger LED Animations. Receives ; commands from ORCΛ
As a side fun idea while learning Nodejs and while working for a bigger and more complex project with my partner Hendrik I programmed Remora just to have an easy way to launch animations from ORCΛ
If you want to try it out please check the requirements and components on the repository or on the hackaday project page. After getting addressable LEDs and any ESP32 board, it’s easy to compile this, reading the provided instructions with Arduino or Plataformio. It needs simple configuration, to know your WiFi credentials and the length of your led stripe. After initializing and getting online, the firmware will listen by default in port 49161 and launch short animations as fast as the UDP message hits the chip. To test it send an UDP message using netcat or any other way. In Linux that can be done like this:
echo '82O'|nc -w1 -u ESP32_IP 49161
IMPORTANT: If you copy the upper text make sure that the quotes are single quotes otherwise the compiled C will not recognize what animation to start
That is enough if you want to send animations to one controller. If you need more then we created a simple UDP redirector script that lives in the extras folder. This can be run using:
After configuring a simple ip-config.json file that has an id to IP lookup table:
When this is running in the background, it will just process any message coming, extract the first character and pick up that ID from the table. Then it will redirect the rest of the message to that IP, same port 49161. To explain it more graphically, sending:
echo '282O'|nc -w1 -u localhost 49161
will lookup IP 2 and send 82O to 192.168.0.73 enabling you to send animations to different LED Stripes from Orca sequencer software.
I wanted to document on this paper my efforts on researching a way to send efficiently a lot of data to an ESP32.
One of the attractive features of UDP is that since it does not need to retransmit lost packets nor does it do any connection setup, sending data incurs less delay.
That made it to be very good choice to transmit audio or video over the Web. But in my case-study, working for a LED stripe hardware controller in team with @hputzek, we where interested in high framerate transmission with the less possible bandwidth use. I first researched in the direction of zlib (miniz) that is already in the ESP32 Core in it’s minimal expression. I could not get it to work since I did not know back then how to convert incoming bytes into their corresponding integer. But here is a solution if someones wants to try the miniz zlib compression/decompression out. It’s already on ESP32 Rom, so I guess it should have less overhead than any other decompression method.
Then looking forward other decompression methods Brotli come in the scene:
Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a great compression ratio
Brotli library definition
The goal was to send a byte for each Red, Green and Blue values of a 144 LEDS Stripe but with the same technology would be possible to send fast samples of complex sensor metrics. Test samples are here available. Samples of 1.45 KB bytes where compressed with zlib (85 bytes) and with Brotli (57 Bytes) giving a clear win over Brotli compression. Speed of decompression was between 1 and 2 ms on the microprocessor. Here is the udp-receive (server) test that I made just as a raw proof-of-concept: https://github.com/martinberlin/brotli/tree/master/tests/udp-receive
Compiling that on a ESP32 it’s possible to test the uncompressed result via Serial sending with any tool (Like netcat) a brotli compressed bytestream to the ESP32 IP:
cat 144.txt.br |nc -w1 -u ESP_IP_ADDRESS 1234
Check also the tests/udp-receive branch for a more advanced example using xTaskCreatePinnedToCore. Seeing this work made us choice Brotli as a decompressor and UDP as transport protocol. Thanks to @bitluni for your suggestion last weekend.
This a sample of the result of reading a fast UDP stream of compressed bytes and sending the output to an addressable Led stripe.
Well actually the second one is for him, but the other that was made for my brother is still there, so we decided to run some tests. One of the cool things I find about ESP-IDF as a framework instead of Arduino framework for Espressif chips is that before compiling you can do a : make menuconfig
So basically you can set the configuration variables before compiling. Not only from main code but also for any component that is loaded using git submodules. This is how the Light example configuration looks like:
So what my father did after I installed him the ESP-IDF / MDF stuff in his Ubuntu laptop is to debug the Lamp logs to see how they behave, to analyze the Root-> Nodes behaviour as described in the Mesh API Guide.
If the root node lamp is disconnected, after no receiving beacon frames that is kind of a heartbeat to prove root node is alive, child Lamps organize a voting session to see who is going to become the next root node. We can see this clearly in this Log:
FIRST LAMP CONNECTS [0;32mI (695) phy: phy_version: 4008, c9ae59f, Jan 25 2019, 16:54:06, 1, 0 [0m 14:40:26.181 -> I (696) wifi: mode : sta (3c:71:bf:a9:45:14) 14:40:26.181 [0;32mI (745) [mwifi, 138]: esp-mdf version: 67a0a5b [0m 14:40:26.181 -> W (745) wifi:[beacon]new interval:100ms 14:40:26.181 -> I (747) wifi: mode : sta (3c:71:bf:a9:45:14) + softAP (3c:71:bf:a9:45:15) 14:40:26.185 -> I (751) wifi: Init max length of beacon: 752/752 MESH IS STARTED 14:40:27.204 -> I (1773) mesh: ;need_scan:0x1, need_scan_router:0x0, look_for_nwk_count:1 14:40:27.204 -> [0;32mI (1774) [mwifi, 98]: MESH is started [0m 14:40:27.204 -> [0;32mI (1778) [light, 655]: event_loop_cb, event: 0x0 [0m 14:40:27.204 -> [0;32mI (1783) [light, 660]: MESH is started [0m I (2075) mesh: find root:ESPM_D9C830, root_cap:2(max:256), new channel:1, old channel:0 14:40:27.510 -> I (2075) mesh: [S2]MiFibra-509A, 4c:1b:86:5a:50:9c, channel:1, rssi:-28 14:40:27.510 -> I (2078) mesh: find router:[ssid_len:12]MiFibra-509A, rssi:-28, 4c:1b:86:5a:50:9c(encrypted), new channel:1, old channel:0 SECOND LAMP IS CONNECTED 14:40:27.544 -> [0;32mI (2125) [light, 683]: the root connects to another router with the same SSID [0m I (2428) mesh: [SCAN][ch:1]AP:6, other(ID:0, RD:0), MAP:1, idle:0, candidate:1, root:1, topMAP:0[c:0,i:0][4c:1b:86:5a:50:9c]router found 14:40:27.850 -> I (2431) mesh: 6545[selection]try rssi_threshold:-78, backoff times:0, max:5 14:40:27.884 -> I (2438) mesh: [DONE]connect to parent:ESPM_D9C830 (This is the hidden SSID from ROOT Node Lamp), channel:1, rssi:-41, 30:ae:a4:d9:c8:31[layer:1, assoc:0], my_vote_num:0/voter_num:0, rc[00:00:00:00:00:00/-120/0] D (3513) [light, 168]: Erase restart count [0m I (3961) wifi: n:1 1, o:1 0, ap:1 1, sta:1 1, prof:1 14:40:29.375 -> I (3963) wifi: state: init -> auth (b0) I (3971) wifi: state: auth -> assoc (0) 14:40:29.409 -> I (3979) wifi: state: assoc -> run (10) 14:40:29.409 -> I (3980) wifi: connected with ESPM_D9C830, channel 1 NOTE: I guess restart count is a counter to reset the Lamp configuration, since turning it on/off for 3 times, resets the Lamp and blinks yellow waiting for configuration. SO NOW, WE DISCONNECT ROOT NODE (First Lamp) The node child has no parent, hence no connectivity at all and does not receive any commands, this is what happens [light, 140]: System information, channel: 1, layer: 2, self mac: 3c:71:bf:a9:45:14, parent bssid: 30:ae:a4:d9:c8:31, parent rssi: -34, node num: 2, free heap: 159120 [light, 655]: event_loop_cb, event: 0x8 [0m 14:44:22.587 -> I (237153) mesh: [wifi]disconnected reason:200(beacon timeout), continuous:1/max:12, non-root, vote(,stopped) 14:44:22.587 -> [0;32mI (237158) [light, 669]: Parent is disconnected on station interface [0m 14:44:22.620 -> I (237168) mesh: [scan]new scanning time:600ms And after this starts the voting to see what Lamp is going to be the next Root lamp.
The interesting part of a Mesh network is that is like a self-healing network, where the hierarchy is redefined once you remove one of it’s pieces, and reconstructs automatically. Of course this takes time, and if you remove a Root node, all your system will be idle for some seconds (in our tests was up to 20/30 seconds) till the selection of a new Root node takes place. But the benefit, is that as every lamp can be connected to it’s parent and at the same time be an Access point where childs connect, you can extend your WiFi reach. Every lamp is like a small WiFi that broadcasts commands to it’s childs.
About a month ago I started learning to use ESP-IDF and got very interested in Espressif Mesh Lamps. It’s a lot of fun for me to create my own ESP-Mesh LEDs lamps with this technology. But as with everything I touch that is open-source, apart of being a user, I try to collaborate and made the thing better. That’s the spirit of open source, you are a user, but also at the same time your opinion counts and usually it’s welcomed.
Comparison from previous ESP-Mesh App and new upcoming version:
03/2019 new version
More integrated and compact UX. Now On/Off switch is in the same screen allowing for easier usability
Doing White with RGB is now possible. Click on the center and the light will turn on the 3 colors at the same time (New feature)
Warm/Cold switch is also better signalized and more usable.
I really like the update and I think is a significant improvement over the last version.
Here some selfish pictures of my last lamp projects ;)
I started to dig a bit more into the Iot development network from Espressif with the intention to slowly learn something new. All the official examples of Mesh lights, ESP32-Camera and ESP-WHO that presents a preliminary version of face recognition are built on the top of the IDF.
So sooner or later, I think will be the official way to do “internet of the things” devices with these Boards. But I’m still not ready for it, I need to do simple examples and experimenting before taking over and do something for real.
So the first things I tackled on is to take this libraries “as is” and use them in existing Arduino framework projects. And it’s fully possible, below is a small example about this using the TTGO Camera with Pir sensor
“ESP–Mesh reduces the loading of smart light devices on the router by forming a mesh with the smart light device.”
I’ve started months ago getting some smart lights for home and I’ve chosen Osram since it was about half the price than Phillips VUE. But the thing with this systems, though they work nice and with very little configuration, is that you need always a “Gateway”. A central point that receives signals and then send via Radio frequency to the lights what they have to do.
The Espressif ESP-Mesh took a different approach: Every device is a network member making a Mesh of interconnected devices. There is no gateway because there is no need for one.
There is also a very important point that is very interesting if you are a maker like me. It’s open source. That means you don’t need to buy a 60€ light to test it, you can just go to the ESP-Mesh github repository and download ESP-Mesh-Light example to compile it in one of your existing ESP32 Boards. Then you can get easily something like this working :
So what I’m working on the free time to take a rest from another pending projects is to take this PWM output and amplify it using a 74HC125 Quad Bus Buffer to power more than one led.
Command lines to execute the compilation are only 2:
make menuconfig make erase_flash flash
Compile this into your ESP32 and then download Espressif official Android App or if you are using I-phone / I-Pad just search for “ESP-Mesh” The Mesh devices are configured using Bluetooth so keep in mind to have this enabled on your device. Instead of taking the “WiFi Manager” Approach, using an App, you have the benefit that you can just send the WiFi Credentials plus configuration directly to the ESP32.
Since beginning of 2019 I’m participating in a project called “Low cost / low power sleep / WiFi Camera” so I purchased a couple of low cost boards, between them the LilyGO ESP 32 Camera.
BigIot.net is an interesting IOT API site
Not only an API but a very interesting infrastructure that is also possible to link BIGIOT with WeChat allowing to “chat” with your device and use it for Smart home or just to check what was the last image uploaded. In my fork of this ESP32 Camera, since the original looks a bit unmaintained without even an issue board, I added some better display information:
This example shows how to set up an automatic camera that takes pictures every 20 seconds and sends them like an API Post push to BigIot. So far so good, it works at first glance. The issue is that BigIOT is all in Chinese and there is no english. So if you consider that a problem then you can stop here. I didn’t because I think the Chinese know hell about this and they are leading the IoT world for a good reason.
So I translated it as I could just to register and get this pieces of information:
After registering you just have to open the activation Email and you are good to go. Just log in the Admin Panel and try to feel at home. Most important parts are:
智能设备列表 List of your Devices -> Add a “Camera device”
数据接口列表 List of your Inputs -> This is the Interface, so just make an “Upload interface”
And that are the two IDs you need, device ID and interface ID. After that and compiling this, the small camera is going to start pushing pictures to BigIOT and you can just click on the stats on the interface area and start checking the pictures when you are away.
Very good to control for example a 3D printer that you leave at home finishing something. This is the result:
https://gitee.com/hejinlv/ESP32/blob/master/ESP32_bigiot_LED/ Here there is a great example on how to Listen to Commands to interact with WeChat
Serial parallel interface was developed by Motorola in the eighties and is the standard or short communication in embedded devices.
The data transmission is explained in detail in the Wikipedia SPI article. But basically is along this lines and we will add some code examples below:
The bus master sets up the clock using a frequency supported by the Slave device.
Master selects the slave device with a logic level 0 on the select line.
During each SPI clock cycle a full duplex data transmission occurs. Master sends a bit on the Master Output Slave In line (MOSI) and the slave reads it. The slave sends a bit on the MISO line and master reads it.
That’s basically the essence of it. There is much more happening under the hood and it’s explained with more in detail on the referenced article. The interesting part of the SPI interface is that we can have many Slaves communicating with the master, since when the SS line is HIGH the slave ignores the master, so we can basically have a different SS (Or Chip select line) per device. So you can use the same Clock, Mosi and Miso to communicate with many SPI devices as long as the data communication happens with one device at the time.
The problem appears when you need to communicate with two different SPI devices at the same time
For example, in the WiFi Camera project, we made an experimental model with a st7735 small 128×128 pixels color TFT display that is also SPI. So if you want to make a progress bar in the moment where we are reading the Camera memory, you cannot use the same SPI interface, because you will share the data line and get a corrupted image.
Maybe there is a way to do it that I’m not aware of. When yes, then please add a code example ;) So my intuitive solution to the problem was to use a second SPI interface, completely independent from the Camera SPI. So that was a nice idea, but implementing it was not so nice, and the issue is that most of the libraries instantiate SPI like this:
SPI.begin(TFT_SCLK, TFT_MISO, TFT_MOSI, SS);
SPI.begin(); // To use default GPIOs
At this point we could make a stop and mention that the ESP32 has some custom SPI gpios that are by default used when you do an SPI.begin() without any values. And that is very well explained in the Espressif documentation for SPI master driver:
IOMUX pins for SPI controllers are as below:
So if I understand well and you see the GPIO writings on most ESP32 boards SPI.begin() without parameters starts the interface using the VSPI gpios also 18,19 and 23. But what happens if you want to use another GPIOS ? No problem, just start SPI.begin() with the GPIOs you want to use. That as long that you don’t need to use a second SPI device. Because when you are instantiating SPI at a global level, you are instantiating a class that will use that GPIOs for all your program. So the solution is to use two different classes and there is a very good example about this in the SPI Multiple buses example on Espressif repository.
The problem is many libraries have not considered the possibility of multiple SPIs
At this point I had a solution but I was still far away of implementing it. Arducam library is pretty old and I could say, not maintained for the future generations of ESP32 chips. Even the SPI instantiation has to be done outside the Lib although the SPI.transfer and so on are fixed inside it. Too bad. TFT st7735 is a very nice lib and well maintained so it lets you define your own SPI gpios in the configuration. So after some minutes of doubt I decided to recode the SPI functions of the Arducam and make them mine. So that is what I did, to recode the SPI functions, leaving arducam library as a minimal I2C communication lib and taking SPI to my own command. Not a big deal, just check what functions are in charge of SPI.transfers and so on, copy them to your own and replace this by your own implementation.
And that is it. After this we will use this hspi
for our camera communication, leaving the other SPI for the display. It
worked out, after some minutes of tweaking and burning my fingers
changing wires from one GPIO to another.