Zonnestroompanelen in Nederland

duurzaamheid achter de meter

(42) ADA – Aduino Dino Animation

by Floris Wouterlood – July 21, 2020

Summary
Stop-motion animations display successive picture frames at such a refreshing speed that one gets the impression of movement. In the cinema more and larger frames are king. In the Arduino world quite the opposite holds true. Frames gobble up memory at alarming speed and the real challenge with Arduinos is to strike a compromise between frame size, number of frames and the available amount of microprocessor memory. Here we run head-on into a hard barrier: the extremely limited amount of memory available in the Arduino. Undaunted by this my aim was to bring a dinosaur to life, ‘Arduino style’, through a series of frames displayed in succession: an ADA. This acronym is also a small tribute to ladyAda, founder of Adafruit Industries and creator of numerous outstanding essential libraries for the Arduino.

The initially goal was to create a 25-frame ADA on a 128*64 LCD powered by an Arduino Uno. A prototype dino animation was tested with several combinations of microprocessor board and display. Most frames (28) were supported by an Arduino Nano – 128*64 OLED combination, while the fastest animation was produced with an Arduino Uno – 480*320 TFT shield with a sketch featuring 26 frames.
This article describes all my technical considerations as well as the procedures followed to prepare an ADA. Three sketches are supplied.

Introduction
All the work in a common Arduino (Uno, Nano) is performed by a single 8-bit processor (ATmega 328) equipped with 1kB EEPROM, 2kB SRAM (‘dynamic memory’) and 32kB flash memory (32,768 bytes of so-called ‘program memory’, of which 30,720 user-addressable). Programs are loaded in program memory while variables are stored in dynamic memory. If one needs more memory the Arduino ‘Mega’ offers an ATmega 2560 microprocessor with 256 kB flash memory and 8kB SRAM. ESP8266 based microboards may carry even more payload, e.g., the Wemos D1 mini has 4 MB of SRAM and 50 kB of flash memory. The standard Arduino’s 30,720 bytes is the physical boundary for our challenge, that is, both the animation’s instructions and the picture frames need to be squeezed into this space.

Figure 1. Frame captured from Moisè Asta’s YouTube movie Velociraptor Animation Test.

Velociraptor
Well known from the Jurassic Park movies is the small, turkey-sized, agile and bipedal dinosaur, Velociraptor. Cinema movie makers have gone far to recreate these animals in the virtual realm although usually movie directors are inclined to present the animals much larger than their actual dimensions. On YouTube a Velociraptor animation test created by Moisè Asta (www.moiseasta.com) with amazing realistic recreation of movement can be enjoyed at https://www.youtube.com/watch?v=Og983D3dTtM. This species has been model for my ADA. One extracted picture frame of Moisè’s study is pictured in figure 1. This frame serves in figure 2 to explain the processing steps necessary to mold the dino into an image format that is manageable by an Arduino.

Displays
The challenge in the current project was to achieve stop-motion animation featuring Velociraptor on the Arduino Uno, and to see how well the animation performs with several displays. For this purpose I had the following displays at hand: 128*64 pixel ST7920 LCD, 128*64 SSD1306 OLED, 130*130 SSD1283A TFT, and 320*240 and 320*480 pixel TFT displays with various ILI controllers. For several reasons explained below the graphical 128*64 LCD for the Arduino was used as the basic display vehicle.

Reduction, reduction, reduction
Any stop-motion animation consists of a sequence of individual frames. Movies downloaded from YouTube take several megabytes of storage. Imagine a 10 MB movie compressed into 32 kB of storage memory in an Arduino. No way. The key to success here is radical reduction of the number of frames, frame size and color depth. Make things small and simple and they will work on the Arduino! This means that in order to animate a dino we need to calculate, nibble and save. Actually, we must nibble and save so much that Ebenezer Scrooge in comparison would be a salutary philanthropist. One single 320*480 frame in 16-bit color takes 307,200 bytes (300kB). After reducing color depth all the way down to monochrome, 1-bit color, one single frame would still require 320*480*1 / 8 = 19,2 kB of memory space. One single frame would require 60% of memory space in our project!

Graphical LCD our basic display vehicle
Because a graphical LCD for the Arduino offers 128 pixels in width and 64 pixels in height and by design works with a color depth of 1 bit (any pixel can be either ON of OFF), this display was selected as standard. One 128*64 pixel image in 1-bit color depth claims 1,024 bytes of memory. A small calculation learns that the Arduino’s accessible program memory (30,720 bytes) completely fills up with 30 frames of 1,024 bytes each. That is the maximum supported number of frames at this resolution and color depth. In real sketches the number of frames accommodated will be lower because we need instructions to start the display, start the animation cycle and to send the frames in fast sequence to the display. The goal therefore was a realistic 25 frames and next to experiment with sketches to probe the maximum duration of an ADA.

Procedure in brief
The principal steps are the following: download movie – extract frames – engage in segmentation – scale down – export frames as monochrome bitmaps – convert these bitmaps into c-arrays – import in Arduino sketch – run animation.

Downloading movies
Downloading YouTube movies to local storage medium is possible with a variety of plugins or extensions to existing browsers. Be aware of the risk that some of these extensions may have adverse features such as sneakily installing search bars, advertisement additions or installing malware right away. Be careful or immediately uninstall/reinstall your browser if unwanted activity is seen. Having a dispensable virtual machine available for his purpose may be handy. Usually movies are in MP4 format but there is a jungle of fancy formats awaiting you in the video world.

Frame extraction
Several programs (e.g., EZGif, VLC, ImageJ) offer functionality to extract frames from movies. In addition there exist on line extraction services. One must experiment here. I managed to extract frames from MP4 movies via an on line service. The output file format for the frames was PNG which is my favorite format. From these frames I selected 25 that cover the types of motion that I wanted to animate: walk, stand and roar.

Segmentation
First a few words about the different ways in which Arduinos support the display of images:

  • One option is that a sketch calls bitmapped images that are stored externally in BMP format on SD card or on some other medium. This is often done with standalone pictures (‘photo-stand’).
  • Another option is calling images in c-array format stored externally, that is, in the same folder as the sketch.
  • Finally, and that’s the option used here because once the sketch is compiled the Arduino can further do the job on its own, is the support of images that are in c-array format embedded in the sketch.

The trick therefore is to convert the frames extracted from movies into c-array files. These c-arrays can then be opened with an ascii editor and copy-pasted into the appropriate sketch.

There are two ways to process an image captured from a movie to c-array format:

  • Raster graphic segmentation: import frames in a bitmap manipulation program and follow up with filtering, image size reduction and export to 1-bit monochrome images. Next, convert these 1-bit images to c-array files with an appropriate conversion program.
  • Vector graphics segmentation: import frames in a vector drawing program, manually or semi-automatically draw the contours of the dino and export these to 1-bit monochrome images. Next, convert the 1-bit images to c-array files with an appropriate conversion program.

I favor the vector graphic way. Vector drawing programs have the tremendous advantage that the end product (contours) are endlessly and reversibly scalable without losing any detail. Scaling in bitmap manipulation programs often produces jagged pixels blocks while the format used here does not allow anti-aliasing. Scaling of bitmaps thus introduces irreversible changes. Another advantage of working with vector contours is that subtle shifting of a dino contour within its frame outline (‘corrective centering’) is fairly easy to perform. Corrective centering is a necessary manipulation to make sure that the final animation runs smoothly. In a raster graphics environment making corrections by shifting a dino a few pixels in the X or Y direction frame by frame, is doable but remains a headache.

Figure 2. Sequence of vector graphics actions necessary to prepare 1-bit images from a movie.

Segmentation procedure
The procedure used to draw contours and further manipulate frames is shown in Figure 2. First, the 25 selected frames were imported in a vector graphics program, each frame in its own layer. The result: a stack of 25 ‘picture frame’ layers, each containing one of the imported movie frames. The next step was to insert an additional layer between every picture frame layer. These layers are the ‘contour’ layers. In each contour layer the closed contour was drawn of the dino visible in the picture layer immediately below it (fig. 2B). Thus, the end result was a vector file containing a stack of 50 layers: picture frame layers alternating with contour layers. The picture frame layers can subsequently be made ‘hidden’ or outright deleted. What then remains is a stack of 25 contour layers (fig. 2D).
One extra (‘bottom’) layer that contains a rectangle with dimensions exactly matching 128*64 pixels (2:1) on export was added to the contour layer stack. Export of this bottom layer together with one of the dino contour layers produced one monochrome 128*64 pixel image frame, ready to be incorporated in the animation.

Alignment to ensure fluent frame-to-frame transition
A vector program allows the user to move elements freely around, in our case any of the 25 dino contours, while keeping each contour neatly in its own contour layer. The dino contours were realigned relative to each other (fig. 2D, also called ‘corrective centering’) by repositioning them slightly. The aim here was to cause a smooth frame-to-frame transition in the final stop-motion animation. In this stage of the exercise we had to make of course maximal use of the space offered by the 128*64 pixel frame rectangle. Remember: every pixel counts!
After the dino contours had been (re) positioned to my satisfaction each dino contour was filled with black color (in figure 2E, I used red fill color for illustration purposes – note that the end product must be black-white). The 128*64 rectangle in the bottom layer was painted white. The thickness of the outline of the rectangle was set at at zero pixels.

Scaling
All the above manipulations could be done without worrying much about size, dimensions, placement and about maintaining a 2:1 image proportion. It is only at the moment of export to monochrome 1-bit images that these issues really matter. For the preparation of c-array files that will be accepted and compiled by the Arduino compiler we must start from monochrome images with a maximum format of 128*64 pixels because the latter is the maximum frame size we can display on a LCD.
At this point the bottom layer in the vector drawing file with the 2:1 frame comes into focus. This rectangle, together with all dino contours must be scaled to such a size that the dimensions of any exported material will be 128*64 pixels. At this point running a few test exports is advised.

Nibble, nibble, nibble and we ended with 112*61 pixels per frame
Now the real nibbling started. With some scaling and fiddling with the 128*64 pixel rectangular frame in the bottom layer of the vector drawing file I managed to produce exported frames with smaller dimensions: 112*64 pixels. The number 112 for the horizontal rows of pixels is important because it is a multiple of eight. C-array files are 8 pixels to the byte, consequently a frame width of 112 pixels translates into a 14 bytes ‘wide’ c-array. Remember: the smaller the images the more frames can be packed into an ADA sketch. 112*64 pixel bitmap monochrome images each take 896 bytes. 25 of such images amount to 25 x 896 = 22,400 bytes. Further shaving, that is clipping off a few lines from the top and bottom of each frame produced output monochrome bitmaps with dimensions 112*61 pixels. The size of one such an exported bitmap is 854 bytes, that is a nice 42 bytes less per frame than a 112*64 bitmap series. On 25 frames this iteration of nibbling resulted in a total of 1,050 bytes saved, i.e., the equivalent of one complete extra frame!

Export
The preferred format of the monochrome export image files is Microsoft’s BMP format. Note that once exported from the vector drawing these bitmap images are no more nicely scalable, that is, if one attempts to do that the dino becomes a jagged stack of big black squares. Don’t expect to do anti-aliasing because introducing intermediate greyscale pixels is not possible because the images are monochrome.

After-export check
An after-export check is appropriate to save disappointment later on. Whether all export actions have been performed successfully can be checked in any drawing program. A fast and conclusive way is to import all frames in imageJ as a sequence and to save this as an animated GIF. Any outlier frame will be rejected by imageJ. This program, also called Fiji is a free, open source, Java based program.

Conversion
The result of all export activity was a set of 25 monochrome BMP images that needed to be converted into as many c-array files. Conversion was done with the program ‘lcd-image-converter.exe’ which is a Windows based program freely available at Sourceforge (https://sourceforge.net/projects/lcd-image-converter/). The alternative is to experiment with an on-line BMP-to-c-array converter. I took the lcd-image converter way (figure 3).

figure 3. Workscreen of LCD-image-comverter. This program is handy to import the 1-bit BMP vector output file and convert that to c-array file. The ‘convert’ step is illustrated.

Work flow in lcd-image-converter is as follows:

  • Import a monochrome bitmap (Image, import),
  • invert (makes a black dino on white background),
  • Convert (File, convert),
  • repeat this procedure with all remaining BMP images.

Output of each cycle is a c-array file in ascii format with extension .c that can be opened in any ascii text editor. I used for this purpose Gedit (Linux) or Notepad++ (Windows). A screen capture of one of the c-array files (edited to shorten the picture) is shown in Figure 4. Note that some libraries require c-arrays in XBM format. This occurred when I tried to run the animation with the combination Wemos D1 mini – <u8g2lib.h> and the 128*64 LCD.

figure 4. Partial content of a c-array file, here that of frame dino_25. The header is just for information and documentation; the static const uint8_t {…..}; section is what we need to include in the sketch (section indicated with green line)

Composition of an output c-array file
A c-array output file produced by lcd-image-conversion.exe contains three sections: header, image and a body of hexadecimal code (figure 4):

  • Header section (with all lines commented out) that contains information about the file and how the original bitmap was processed by lcd-image-conversion,
  • Image section which contains commented-out lines with dots and blocks that provide a rough visual impression in ascii format of the 1-bit image being displayed. Notice that there are 61 lines with each 112 dots or blocks, in other words a representation of what is going to be displayed on screen. To save space in figure 4 this section is represented by a scaled-down ‘dino’ image,
  • Bottom section that is the most valuable part of the file: the hex array code. This is the section that matters. Pixels are coded as bytes, thus a 112 pixel wide image will produce 112 : 8 = 14 columns of bytes. The number of rows is 61 or equal to the height of the image.

The c-array starts with a variable: static const uint8_t, followed by the image’s file name and size (between the brackets). File size of each of the 25 c-array files is 854 bytes. Together they will animate into 25 frames that together require 25 x 854 = 21,350 bytes of program memory.

Clip ‘n shave, and even more clip produces more frames
Each frame now measured 112 pixels wide and 61 pixels high (854 bytes), allowing at least one and perhaps two extra frames compared with 112*64 frames! A 112*61 pixels ‘dummy’ frame was prepared consisting of 61 lines of 12 bytes coded 0x00 (8 OFF pixels in each byte) and included in the animation.
With all the contour drawing, export actions, nibbling, clipping and saving I had now ready for the ADA a set of 25 ‘dino’ c-arrays plus one ‘dummy’. These were copy-pasted into the final sketch. With the combination of Arduino Nano – 128*64 LCD there still was unused program memory. Total available program memory in his case allowed 28 frames at 112*61 pixels. The extra three frames were prepared by going back to the vector drawing file and creating one intermediary ‘dino’ file, and by calling the dummy frame twice. This particular 28-frame ADA uses 29,864 bytes (97% of parogram storage space). As I started this project with an expectation of getting at most 25 frames from the downloaded dino animation study movie I was in fact overwhelmed by success – the studio running out of available dino frames!

Sketch to run the animation on a 128*64 ST7920 LCD / Arduino Nano
Discussed here is a sketch written for an Arduino Nano that provides output to a 128*64 LCD with ST7920 controller. C-arrays are defined in program memory as static const unsigned chars, this is necessary to spare dynamic memory. The c-array that codes for the first animation frame then is as follows:

static const unsigned char dino_00 [ ] PROGMEM = {
followed by the hexadecimal byte array and closed with };

Use of PROGMEM
Reason for using PROGMEM is that if an array is defined in a sketch it will by default be placed by the Arduino IDE compiler in dynamic memory. As shown above, each frame of our animation requires 854 bytes. Dynamic memory (2 kB) would overflow rapidly! To force placement of c-arrays in program memory the library <avr/pgmspace.h> is required. This library demands that c-arrays be defined as a static const unsigned char rather than static const uint8_t.

#include <avr/pgmspace.h>
#include <u8glib.h>

U8GLIB_ST7920_128X64 u8g (13, 11, 12, U8G_PIN_NONE); // constructor

int delaytime = 10; // recommended for slow LCDs

static const unsigned char dino_00 [ ] PROGMEM = {

0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // 14 columns, 61 rows
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,

( …., …., …., etc)

0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0xf8, 0x00, 0x00, 0x00, 0x00,
0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x78, 0x00, 0x00, 0x00, 0x00 // row 61
};

Each byte codes for 8 pixels. To save space here only four of the total of 61 rows of hexadecimally coded bytes are shown.

Dino on LCD display – sketch #1
The sketch ‘ADA_LCD.ino’ has 28 frames. This sketch was created for:

Arduino Uno/ Nano driving a 128*64 LCD with ST7920 controller
library: <u8glib.h>

The library required to work with the 128*64 LCD is <u8glib.h> written by Oliver Kraus. In the dino sketch, void setup(void) is empty while all action is in void draw(void).

Dino on OLED displays – sketch #2
128*64 monochrome OLED displays are becoming increasingly popular. They are cheap and easy to implement via SPI or i2c interfaces.

The sketch ‘ADA_OLED.ino’ has 26 frames. It was created for:

Arduino Uno/ Nano driving a 128*64 i2c OLED with SS1306 controller
library: <u8glib.h>

Pushing the colors to support TFT displays – sketch #3
TFT display shields that fit the Arduino Uno are attractive because they can display 16-bit color. These shields are available with displays in a range of sizes and resolutions. Here we deal with a 2.8 inch 320*240 TFT shield with a ILI9341 controller on board.

The sketch ‘ADA_TFT.ino’ has 25 frames. It was created for:

Arduino Uno driving a 320*240 TFT with ILI9341 controller
library: <mcufriend_kbv.h>

The standard way to send images to the display is by issuing an instruction such as ‘drawBitmap’, followed by coordinates, size of the image and the label under which that image is stored in program memory. Apparently this instruction starts in <u8glib.h> a routine that first collects the image, stores it in a buffer and then dumps the content of the buffer to screen.
A routine faster than buffering is to pick up bytes that belong to a frame directly from their location in program memory and direct them immediately to the display. This saves buffering time and thus produces a faster animation. With some TFT displays, notably those supported by David Prentice’s <mcufriend_kbv.h> library this direct method of frame buildup is possible. The key instruction here is ‘pushColor’. This instruction works in fact so efficient that insertion of a delay is necessary between each frame to prevent the animation from running too fast! A disadvantage of a ‘pushColor’ subroutine in a sketch is that this procedure consumes a certain amount of program memory which with several TFT displays conflicts with the size of the ADA. In those cases the dummy frame or the dummy frame plus the first dino frame were sacrificed. Below follows a list of tested TFT displays together with the microprocessor board to which these displays were wired and the number of frames supported.

pushColor method (<mcufriend_kbv>) – shields

  • TFT shield 2. 4inch ILI 9481 320*240 (on UNO) = 25 frames, if more frames then a avr dude verification error occurred during upload.
    30,252 bytes of program memory space with 25 frames
  • TFT shield 2.8 inch ILI 9341 320*240 (on UNO) = 26 frames, sketch uses 31,146 bytes of program memory space with all frames.
  • TFT shield 3.5 inch ILI 9486 320*480 (on UNO) = 26 frames, sketch uses 31,128 bytes of program memory space with all frames.
  • TFT shield 3.95 inch ILI 9488 320*480 (on UNO) = 26 frames, sketch uses 31,112 bytes of program memory space with all frames.

pushColor method with breakout TFT

  • TFT breakout 1.6 inch (130*130) SSD1283A 1.6 inch on Nano = 24 frames. Sketch consumes 30,144 bytes of program memory space.

128*64 OLED

  • 7-pins SSD1306 monochrome OLED on Nano = 26 frames, sketch uses 27,726 bytes
  • 4-pins SSD1306 i2c monochrome OLED on Nano = 28 frames (dummy frame called twice plus an intermediary ‘dino’ frame), sketch uses 29,516 bytes.

More frames = different processor
As it turns out, an Arduino needs less overhead programming to drive a 128*64 LCD display than other combinations of microprocessor / display do and therefore can show a record number of 28 frames. For instance, with a Wemos D1 mini with ESP8266 processor and a 128*64 LCD under the <u8g2lib.h> library I could manage ‘only’ 25 frames. The <u8g2lib.h> library was necessary here because the compiler servicing the Wemos D1 mini does not support <u8glib.h>.

Image manipulation software used in this project

Downloadable sketches (packed into one zip file named ADA-sketches.zip)

ADA_LCD.ino – Arduino Uno plus ST7290 128*64 graphical LCD (28 frames)
ADA_OLED.ino – Arduino Uno plus SSD1306 i2c 4-pin 128*64 monochrome OLED (26 frames)
ADA_TFT.ino – Arduino Uno plus ILI9341 shield 320×240 TFT (26 frames)

download ADA-sketches.zip

Enjoy!