Custom MicroPython#

The WiFi co-processor requires two custom C modules:

  • MessagePack is an efficient binary serializer available for many languages, and

  • FinaliserProxy, a helper class for MicroPython to call the finaliser of user-defined Python objects.

Let’s add them to the stock MicroPython interpreter.

The instructions below are for an esp32 wifi server (called wifi-server) and an stm32 client (wifi-client).

Server#

Install/update the local MicroPython branch:

%%bash

# interpreter
cd $IOT/mp
if [ ! -d micropython ]
then
    git clone git@github.com:micropython/micropython.git
else
    cd micropython
    git checkout master
    git pull
    git merge master
fi

# library
cd $IOT/mp
if [ ! -d micropython-lib ]
then
    git clone git@github.com:micropython/micropython-lib.git
else
    cd micropython-lib
    git checkout master
    git pull
    git merge master
fi
Hide code cell output 
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
Already up to date.
Cloning into 'micropython-lib'...

Download the custom modules:

%%bash

cd $IOT
svn checkout https://github.com/iot49/iot49.org/trunk/mp
Hide code cell output 
A    mp/boards
A    mp/boards/ADAFRUIT_F405_IOT
A    mp/boards/ADAFRUIT_F405_IOT/bdev.c
A    mp/boards/ADAFRUIT_F405_IOT/board.json
A    mp/boards/ADAFRUIT_F405_IOT/board_init.c
A    mp/boards/ADAFRUIT_F405_IOT/mpconfigboard.h
A    mp/boards/ADAFRUIT_F405_IOT/mpconfigboard.mk
A    mp/boards/ADAFRUIT_F405_IOT/pins.csv
A    mp/boards/ADAFRUIT_F405_IOT/stm32f4xx_hal_conf.h
A    mp/modules
A    mp/modules/finaliserproxy
A    mp/modules/finaliserproxy/finaliserproxy.c
A    mp/modules/finaliserproxy/micropython.cmake
A    mp/modules/finaliserproxy/micropython.mk
A    mp/modules/ioctl
A    mp/modules/ioctl/ioctl.c
A    mp/modules/ioctl/micropython.cmake
A    mp/modules/ioctl/micropython.mk
A    mp/modules/micropython.cmake
A    mp/modules/msgpack
A    mp/modules/msgpack/micropython.cmake
A    mp/modules/msgpack/micropython.mk
A    mp/modules/msgpack/msgpack.c
Checked out revision 150.

Find the port of the connected microcontroller (for flashing):

%connect wifi-server

Connected to wifi-server @ serial:///dev/ttyUSB0

Update the port (last line) in the cell below, then run it compile and flash the custom MicroPython interpreter with the new modules:

%%service esp-idf

cd $IOT/mp/micropython/ports/esp32
make submodules
make BOARD=GENERIC_OTA clean 
make BOARD=GENERIC_OTA \
     FROZEN_MANIFEST=../../../boards/manifest_release.py \
     USER_C_MODULES=../../../../modules/micropython.cmake \
     PORT=/dev/ttyUSB0 deploy
Hide code cell output 
setting up IDF ...
idf.py -D MICROPY_BOARD=GENERIC_OTA -B build-GENERIC_OTA  -DUSER_C_MODULES=../../../../modules/micropython.cmake -D MICROPY_FROZEN_MANIFEST=../../../boards/manifest_release.py -p /dev/ttyUSB0 -b 460800 flash
[1/216] cd /home/iot/iot49.org/mp/micropython/ports/esp32/build-GENERIC_OTA/esp-idf/main && echo -n
[2/215] Performing build step for 'bootloader'
ninja: no work to do.
[3/213] Generating ../../genhdr/moduledefs.h
[4/213] Generating ../../genhdr/qstr.i.last
[5/213] Generating ../../genhdr/qstr.split
[6/213] Generating ../../genhdr/qstrdefs.collected.h
QSTR not updated
[7/14] Building C object esp-idf/main/CMakeFiles/__idf_main.dir/home/iot/iot49.org/mp/micropython/py/objmodule.c.obj
[8/14] Building C object esp-idf/main/CMakeFiles/__idf_main.dir/home/iot/iot49.org/mp/modules/ioctl/ioctl.c.obj
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c: In function 'ioctl':
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c:39:57: warning: passing argument 3 of 'stream_p->ioctl' makes integer from pointer without a cast [-Wint-conversion]
     mp_int_t ret = stream_p->ioctl(stream_obj, request, &flags, &errcode);
                                                         ^~~~~~
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c:39:57: note: expected 'uintptr_t' {aka 'unsigned int'} but argument is of type 'mp_uint_t *' {aka 'unsigned int *'}
[9/14] Linking CXX static library esp-idf/main/libmain.a
[10/14] Generating ld/esp32.project.ld
[11/14] Building C object CMakeFiles/micropython.elf.dir/home/iot/iot49.org/mp/modules/ioctl/ioctl.c.obj
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c: In function 'ioctl':
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c:39:57: warning: passing argument 3 of 'stream_p->ioctl' makes integer from pointer without a cast [-Wint-conversion]
     mp_int_t ret = stream_p->ioctl(stream_obj, request, &flags, &errcode);
                                                         ^~~~~~
/home/iot/iot49.org/mp/modules/ioctl/ioctl.c:39:57: note: expected 'uintptr_t' {aka 'unsigned int'} but argument is of type 'mp_uint_t *' {aka 'unsigned int *'}
[12/14] Linking CXX executable micropython.elf
[13/14] Generating binary image from built executable
esptool.py v3.0
Generated /home/iot/iot49.org/mp/micropython/ports/esp32/build-GENERIC_OTA/micropython.bin
[13/14] cd /opt/esp/idf/components/esptool_py && /opt/esp/tools/cmake/3.16.4/bin/cmake -D IDF_PATH="/opt/esp/idf" -D ESPTOOLPY="/opt/esp/python_env/idf4.2_py3.7_env/bin/python /opt/esp/idf/components/esptool_py/esptool/esptool.py --chip esp32" -D ESPTOOL_ARGS="--before=default_reset --after=hard_reset write_flash @flash_args" -D WORKING_DIRECTORY="/home/iot/iot49.org/mp/micropython/ports/esp32/build-GENERIC_OTA" -P /opt/esp/idf/components/esptool_py/run_esptool.cmake
esptool.py --chip esp32 -p /dev/ttyUSB0 -b 460800 --before=default_reset --after=hard_reset write_flash --flash_mode dio --flash_freq 40m --flash_size 4MB 0x8000 partition_table/partition-table.bin 0xd000 ota_data_initial.bin 0x1000 bootloader/bootloader.bin 0x10000 micropython.bin
esptool.py v3.0
Serial port /dev/ttyUSB0
Connecting....
Chip is ESP32-D0WDQ6 (revision 1)
Features: WiFi, BT, Dual Core, Coding Scheme None
Crystal is 40MHz
MAC: 30:ae:a4:30:80:14
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Compressed 3072 bytes to 139...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (139 compressed) at 0x00008000 in 0.0 seconds (effective 4991.5 kbit/s)...
Hash of data verified.
Compressed 8192 bytes to 31...
Writing at 0x0000d000... (100 %)
Wrote 8192 bytes (31 compressed) at 0x0000d000 in 0.0 seconds (effective 25924.5 kbit/s)...
Hash of data verified.
Compressed 21216 bytes to 13403...
Writing at 0x00001000... (100 %)
Wrote 21216 bytes (13403 compressed) at 0x00001000 in 0.3 seconds (effective 528.8 kbit/s)...
Hash of data verified.
Compressed 1484832 bytes to 1001629...
Writing at 0x00010000... (1 %)
Writing at 0x00014000... (3 %)
Writing at 0x00018000... (4 %)
Writing at 0x0001c000... (6 %)
Writing at 0x00020000... (8 %)
Writing at 0x00024000... (9 %)
Writing at 0x00028000... (11 %)
Writing at 0x0002c000... (12 %)
Writing at 0x00030000... (14 %)
Writing at 0x00034000... (16 %)
Writing at 0x00038000... (17 %)
Writing at 0x0003c000... (19 %)
Writing at 0x00040000... (20 %)
Writing at 0x00044000... (22 %)
Writing at 0x00048000... (24 %)
Writing at 0x0004c000... (25 %)
Writing at 0x00050000... (27 %)
Writing at 0x00054000... (29 %)
Writing at 0x00058000... (30 %)
Writing at 0x0005c000... (32 %)
Writing at 0x00060000... (33 %)
Writing at 0x00064000... (35 %)
Writing at 0x00068000... (37 %)
Writing at 0x0006c000... (38 %)
Writing at 0x00070000... (40 %)
Writing at 0x00074000... (41 %)
Writing at 0x00078000... (43 %)
Writing at 0x0007c000... (45 %)
Writing at 0x00080000... (46 %)
Writing at 0x00084000... (48 %)
Writing at 0x00088000... (50 %)
Writing at 0x0008c000... (51 %)
Writing at 0x00090000... (53 %)
Writing at 0x00094000... (54 %)
Writing at 0x00098000... (56 %)
Writing at 0x0009c000... (58 %)
Writing at 0x000a0000... (59 %)
Writing at 0x000a4000... (61 %)
Writing at 0x000a8000... (62 %)
Writing at 0x000ac000... (64 %)
Writing at 0x000b0000... (66 %)
Writing at 0x000b4000... (67 %)
Writing at 0x000b8000... (69 %)
Writing at 0x000bc000... (70 %)
Writing at 0x000c0000... (72 %)
Writing at 0x000c4000... (74 %)
Writing at 0x000c8000... (75 %)
Writing at 0x000cc000... (77 %)
Writing at 0x000d0000... (79 %)
Writing at 0x000d4000... (80 %)
Writing at 0x000d8000... (82 %)
Writing at 0x000dc000... (83 %)
Writing at 0x000e0000... (85 %)
Writing at 0x000e4000... (87 %)
Writing at 0x000e8000... (88 %)
Writing at 0x000ec000... (90 %)
Writing at 0x000f0000... (91 %)
Writing at 0x000f4000... (93 %)
Writing at 0x000f8000... (95 %)
Writing at 0x000fc000... (96 %)
Writing at 0x00100000... (98 %)
Writing at 0x00104000... (100 %)
Wrote 1484832 bytes (1001629 compressed) at 0x00010000 in 24.4 seconds (effective 487.4 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Executing action: flash
Running ninja in directory /home/iot/iot49.org/mp/micropython/ports/esp32/build-GENERIC_OTA
Executing "ninja flash"...
Done

Client#

Now let’s add msgpack to the client. In this example I am using an STM32F405 from Adafruit.

The default board description that comes with MicroPython, ADAFRUIT_F405_EXPRESS, stores the file system in the internal flash of the processor with limited space available. The custom version, ADAFRUIT_F405_IOT, instead uses the external QSPI flash with 2MBytes capacity. It also disables including the usocket and network modules which the wifi coprocessor software replaces.

Change the BOARD variable if you use a different processor.

%%service arm32

cd $IOT/mp/micropython/ports/stm32
cp -rf ../../../boards/ADAFRUIT_F405_IOT boards
make submodules
make BOARD=ADAFRUIT_F405_IOT clean
make BOARD=ADAFRUIT_F405_IOT \
     USER_C_MODULES=../../../modules
Hide code cell output 
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
Updating submodules: lib/libhydrogen lib/lwip lib/mbedtls lib/stm32lib
Synchronizing submodule url for '../../lib/libhydrogen'
Synchronizing submodule url for '../../lib/lwip'
Synchronizing submodule url for '../../lib/mbedtls'
Synchronizing submodule url for '../../lib/stm32lib'
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
rm -rf build-ADAFRUIT_F405_IOT 
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
Including User C Module from ../../../modules/finaliserproxy
Including User C Module from ../../../modules/msgpack
mkdir -p build-ADAFRUIT_F405_IOT/genhdr
GEN build-ADAFRUIT_F405_IOT/genhdr/pins.h
GEN stmconst build-ADAFRUIT_F405_IOT/modstm_qstr.h
GEN build-ADAFRUIT_F405_IOT/genhdr/pybcdc.inf
GEN build-ADAFRUIT_F405_IOT/genhdr/pybcdc_inf.h
GEN build-ADAFRUIT_F405_IOT/genhdr/pllfreqtable.h
GEN build-ADAFRUIT_F405_IOT/genhdr/mpversion.h
GEN build-ADAFRUIT_F405_IOT/genhdr/moduledefs.h
GEN build-ADAFRUIT_F405_IOT/genhdr/qstr.i.last
GEN build-ADAFRUIT_F405_IOT/genhdr/qstr.split
GEN build-ADAFRUIT_F405_IOT/genhdr/qstrdefs.collected.h
QSTR updated
GEN build-ADAFRUIT_F405_IOT/genhdr/qstrdefs.generated.h
GEN build-ADAFRUIT_F405_IOT/genhdr/compressed.split
GEN build-ADAFRUIT_F405_IOT/genhdr/compressed.collected
Compressed data updated
GEN build-ADAFRUIT_F405_IOT/genhdr/compressed.data.h
mkdir -p build-ADAFRUIT_F405_IOT/boards/ADAFRUIT_F405_IOT/
mkdir -p build-ADAFRUIT_F405_IOT/build-ADAFRUIT_F405_IOT/
mkdir -p build-ADAFRUIT_F405_IOT/drivers/bus/
mkdir -p build-ADAFRUIT_F405_IOT/drivers/dht/
mkdir -p build-ADAFRUIT_F405_IOT/drivers/memory/
mkdir -p build-ADAFRUIT_F405_IOT/extmod/
mkdir -p build-ADAFRUIT_F405_IOT/finaliserproxy/
mkdir -p build-ADAFRUIT_F405_IOT/lib/libm/
mkdir -p build-ADAFRUIT_F405_IOT/lib/littlefs/
mkdir -p build-ADAFRUIT_F405_IOT/lib/oofatfs/
mkdir -p build-ADAFRUIT_F405_IOT/lib/stm32lib/CMSIS/STM32F4xx/Source/Templates/
mkdir -p build-ADAFRUIT_F405_IOT/lib/stm32lib/CMSIS/STM32F4xx/Source/Templates/gcc/
mkdir -p build-ADAFRUIT_F405_IOT/lib/stm32lib/STM32F4xx_HAL_Driver/Src/
mkdir -p build-ADAFRUIT_F405_IOT/msgpack/
mkdir -p build-ADAFRUIT_F405_IOT/py/
mkdir -p build-ADAFRUIT_F405_IOT/shared/libc/
mkdir -p build-ADAFRUIT_F405_IOT/shared/netutils/
mkdir -p build-ADAFRUIT_F405_IOT/shared/readline/
mkdir -p build-ADAFRUIT_F405_IOT/shared/runtime/
mkdir -p build-ADAFRUIT_F405_IOT/shared/timeutils/
mkdir -p build-ADAFRUIT_F405_IOT/usbdev/class/src/
mkdir -p build-ADAFRUIT_F405_IOT/usbdev/core/src/
CC ../../py/mpstate.c
CC ../../py/nlr.c
CC ../../py/nlrx86.c
CC ../../py/nlrx64.c
CC ../../py/nlrthumb.c
CC ../../py/nlraarch64.c
CC ../../py/nlrpowerpc.c
CC ../../py/nlrxtensa.c
CC ../../py/nlrsetjmp.c
CC ../../py/malloc.c
CC ../../py/gc.c
CC ../../py/pystack.c
CC ../../py/qstr.c
CC ../../py/vstr.c
CC ../../py/mpprint.c
CC ../../py/unicode.c
CC ../../py/mpz.c
CC ../../py/reader.c
CC ../../py/lexer.c
CC ../../py/parse.c
CC ../../py/scope.c
CC ../../py/compile.c
CC ../../py/emitcommon.c
CC ../../py/emitbc.c
CC ../../py/asmbase.c
CC ../../py/asmx64.c
CC ../../py/emitnx64.c
CC ../../py/asmx86.c
CC ../../py/emitnx86.c
CC ../../py/asmthumb.c
CC ../../py/emitnthumb.c
CC ../../py/emitinlinethumb.c
CC ../../py/asmarm.c
CC ../../py/emitnarm.c
CC ../../py/asmxtensa.c
CC ../../py/emitnxtensa.c
CC ../../py/emitinlinextensa.c
CC ../../py/emitnxtensawin.c
CC ../../py/formatfloat.c
CC ../../py/parsenumbase.c
CC ../../py/parsenum.c
CC ../../py/emitglue.c
CC ../../py/persistentcode.c
CC ../../py/runtime.c
CC ../../py/runtime_utils.c
CC ../../py/scheduler.c
CC ../../py/nativeglue.c
CC ../../py/pairheap.c
CC ../../py/ringbuf.c
CC ../../py/stackctrl.c
CC ../../py/argcheck.c
CC ../../py/warning.c
CC ../../py/profile.c
CC ../../py/map.c
CC ../../py/obj.c
CC ../../py/objarray.c
CC ../../py/objattrtuple.c
CC ../../py/objbool.c
CC ../../py/objboundmeth.c
CC ../../py/objcell.c
CC ../../py/objclosure.c
CC ../../py/objcomplex.c
CC ../../py/objdeque.c
CC ../../py/objdict.c
CC ../../py/objenumerate.c
CC ../../py/objexcept.c
CC ../../py/objfilter.c
CC ../../py/objfloat.c
CC ../../py/objfun.c
CC ../../py/objgenerator.c
CC ../../py/objgetitemiter.c
CC ../../py/objint.c
CC ../../py/objint_longlong.c
CC ../../py/objint_mpz.c
CC ../../py/objlist.c
CC ../../py/objmap.c
CC ../../py/objmodule.c
CC ../../py/objobject.c
CC ../../py/objpolyiter.c
CC ../../py/objproperty.c
CC ../../py/objnone.c
CC ../../py/objnamedtuple.c
CC ../../py/objrange.c
CC ../../py/objreversed.c
CC ../../py/objset.c
CC ../../py/objsingleton.c
CC ../../py/objslice.c
CC ../../py/objstr.c
CC ../../py/objstrunicode.c
CC ../../py/objstringio.c
CC ../../py/objtuple.c
CC ../../py/objtype.c
CC ../../py/objzip.c
CC ../../py/opmethods.c
CC ../../py/sequence.c
CC ../../py/stream.c
CC ../../py/binary.c
CC ../../py/builtinimport.c
CC ../../py/builtinevex.c
CC ../../py/builtinhelp.c
CC ../../py/modarray.c
CC ../../py/modbuiltins.c
CC ../../py/modcollections.c
CC ../../py/modgc.c
CC ../../py/modio.c
CC ../../py/modmath.c
CC ../../py/modcmath.c
CC ../../py/modmicropython.c
CC ../../py/modstruct.c
CC ../../py/modsys.c
CC ../../py/moduerrno.c
CC ../../py/modthread.c
CC ../../py/vm.c
CC ../../py/bc.c
CC ../../py/showbc.c
CC ../../py/repl.c
CC ../../py/smallint.c
CC ../../py/frozenmod.c
CC ../../extmod/moduasyncio.c
CC ../../extmod/moductypes.c
CC ../../extmod/modujson.c
CC ../../extmod/modure.c
CC ../../extmod/moduzlib.c
CC ../../extmod/moduheapq.c
CC ../../extmod/modutimeq.c
CC ../../extmod/moduhashlib.c
CC ../../extmod/moducryptolib.c
CC ../../extmod/modubinascii.c
CC ../../extmod/virtpin.c
CC ../../extmod/machine_bitstream.c
CC ../../extmod/machine_mem.c
CC ../../extmod/machine_pinbase.c
CC ../../extmod/machine_signal.c
CC ../../extmod/machine_pulse.c
CC ../../extmod/machine_pwm.c
CC ../../extmod/machine_i2c.c
CC ../../extmod/machine_spi.c
CC ../../extmod/modbluetooth.c
CC ../../extmod/modussl_axtls.c
CC ../../extmod/modussl_mbedtls.c
CC ../../extmod/moduplatform.c
CC ../../extmod/modurandom.c
CC ../../extmod/moduselect.c
CC ../../extmod/moduwebsocket.c
CC ../../extmod/modwebrepl.c
CC ../../extmod/modframebuf.c
CC ../../extmod/vfs.c
CC ../../extmod/vfs_blockdev.c
CC ../../extmod/vfs_reader.c
CC ../../extmod/vfs_posix.c
CC ../../extmod/vfs_posix_file.c
CC ../../extmod/vfs_fat.c
CC ../../extmod/vfs_fat_diskio.c
CC ../../extmod/vfs_fat_file.c
CC ../../extmod/vfs_lfs.c
CC ../../extmod/utime_mphal.c
CC ../../extmod/uos_dupterm.c
CC ../../shared/libc/abort_.c
CC ../../shared/libc/printf.c
MPY uasyncio/__init__.py
MPY uasyncio/core.py
MPY uasyncio/event.py
MPY uasyncio/funcs.py
MPY uasyncio/lock.py
MPY uasyncio/stream.py
MPY dht.py
MPY lcd160cr.py
MPY lcd160cr_test.py
MPY onewire.py
GEN build-ADAFRUIT_F405_IOT/frozen_content.c
CC build-ADAFRUIT_F405_IOT/frozen_content.c
CC ../../lib/libm/math.c
CC ../../lib/libm/acoshf.c
CC ../../lib/libm/asinfacosf.c
CC ../../lib/libm/asinhf.c
CC ../../lib/libm/atan2f.c
CC ../../lib/libm/atanf.c
CC ../../lib/libm/atanhf.c
CC ../../lib/libm/ef_rem_pio2.c
CC ../../lib/libm/erf_lgamma.c
CC ../../lib/libm/fmodf.c
CC ../../lib/libm/kf_cos.c
CC ../../lib/libm/kf_rem_pio2.c
CC ../../lib/libm/kf_sin.c
CC ../../lib/libm/kf_tan.c
CC ../../lib/libm/log1pf.c
CC ../../lib/libm/nearbyintf.c
CC ../../lib/libm/roundf.c
CC ../../lib/libm/sf_cos.c
CC ../../lib/libm/sf_erf.c
CC ../../lib/libm/sf_frexp.c
CC ../../lib/libm/sf_ldexp.c
CC ../../lib/libm/sf_modf.c
CC ../../lib/libm/sf_sin.c
CC ../../lib/libm/sf_tan.c
CC ../../lib/libm/wf_lgamma.c
CC ../../lib/libm/wf_tgamma.c
CC ../../lib/libm/thumb_vfp_sqrtf.c
CC ../../shared/libc/string0.c
CC ../../shared/netutils/dhcpserver.c
CC ../../shared/netutils/netutils.c
CC ../../shared/netutils/trace.c
CC ../../shared/readline/readline.c
CC ../../shared/runtime/gchelper_native.c
CC ../../shared/runtime/interrupt_char.c
CC ../../shared/runtime/mpirq.c
CC ../../shared/runtime/pyexec.c
CC ../../shared/runtime/stdout_helpers.c
CC ../../shared/runtime/sys_stdio_mphal.c
CC ../../shared/timeutils/timeutils.c
CC ../../extmod/modonewire.c
CC ../../extmod/modnetwork.c
CC ../../extmod/modusocket.c
CC ../../drivers/bus/softspi.c
CC ../../drivers/bus/softqspi.c
CC ../../drivers/memory/spiflash.c
CC ../../drivers/dht/dht.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_adc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_adc_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_cortex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_dma.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_flash.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_flash_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_gpio.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_i2c.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_pcd.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_pcd_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_pwr.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_pwr_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rcc_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rtc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_rtc_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_spi.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_tim.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_tim_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_uart.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_ll_utils.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_ll_usb.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_sd.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_ll_sdmmc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_ll_fmc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_mmc.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_sdram.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_dma_ex.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_dcmi.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_can.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_i2s.c
CC ../../lib/stm32lib/STM32F4xx_HAL_Driver/Src/stm32f4xx_hal_i2s_ex.c
CC usbdev/core/src/usbd_core.c
CC usbdev/core/src/usbd_ctlreq.c
CC usbdev/core/src/usbd_ioreq.c
CC usbdev/class/src/usbd_cdc_msc_hid.c
CC usbdev/class/src/usbd_msc_bot.c
CC usbdev/class/src/usbd_msc_scsi.c
CC boardctrl.c
CC main.c
CC stm32_it.c
CC usbd_conf.c
CC usbd_desc.c
CC usbd_cdc_interface.c
CC usbd_hid_interface.c
CC usbd_msc_interface.c
CC mphalport.c
CC mpnetworkport.c
CC mpthreadport.c
CC irq.c
CC pendsv.c
CC systick.c
CC softtimer.c
CC powerctrl.c
CC powerctrlboot.c
CC rfcore.c
CC pybthread.c
CC factoryreset.c
CC timer.c
CC led.c
CC pin.c
CC pin_defs_stm32.c
CC pin_named_pins.c
CC bufhelper.c
CC dma.c
CC i2c.c
CC pyb_i2c.c
CC spi.c
CC pyb_spi.c
CC qspi.c
CC uart.c
CC can.c
CC fdcan.c
CC pyb_can.c
CC usb.c
CC wdt.c
CC eth.c
CC gccollect.c
CC help.c
CC machine_adc.c
CC machine_bitstream.c
CC machine_i2c.c
CC machine_i2s.c
CC machine_spi.c
CC machine_timer.c
CC machine_uart.c
CC modmachine.c
CC modpyb.c
CC modstm.c
CC moduos.c
CC modutime.c
CC network_lan.c
CC extint.c
CC usrsw.c
CC rng.c
CC rtc.c
CC flash.c
CC flashbdev.c
CC spibdev.c
CC storage.c
CC sdcard.c
CC sdram.c
CC fatfs_port.c
CC lcd.c
CC accel.c
CC servo.c
CC dac.c
CC adc.c
CC boards/ADAFRUIT_F405_IOT/board_init.c
CC boards/ADAFRUIT_F405_IOT/bdev.c
AS ../../lib/stm32lib/CMSIS/STM32F4xx/Source/Templates/gcc/startup_stm32f405xx.s
CC ../../lib/stm32lib/CMSIS/STM32F4xx/Source/Templates/system_stm32f4xx.c
CC system_stm32.c
AS resethandler.s
AS ../../shared/runtime/gchelper_m3.s
CC ../../../modules/finaliserproxy/finaliserproxy.c
CC ../../../modules/msgpack/msgpack.c
CC ../../lib/oofatfs/ff.c
CC ../../lib/oofatfs/ffunicode.c
CC ../../lib/littlefs/lfs2.c
CC ../../lib/littlefs/lfs2_util.c
CC build-ADAFRUIT_F405_IOT/pins_ADAFRUIT_F405_IOT.c
LINK build-ADAFRUIT_F405_IOT/firmware.elf
   text	   data	    bss	    dec	    hex	filename
 357748	     16	  31448	 389212	  5f05c	build-ADAFRUIT_F405_IOT/firmware.elf
GEN build-ADAFRUIT_F405_IOT/firmware0.bin
GEN build-ADAFRUIT_F405_IOT/firmware1.bin
GEN build-ADAFRUIT_F405_IOT/firmware.dfu
GEN build-ADAFRUIT_F405_IOT/firmware.hex

Flash the custom interpreter to the target board.

Verify#

Now check out the new modules on the wifi server and client.

MessagePack#

An example … note that the packed version is quite a bit smaller than the text representation. Msgpack also handles more types than json, for example bytes.

%%connect wifi-server wifi-client

import msgpack, json
from io import BytesIO

# msgpack takes a "stream-like" target
buf = BytesIO()

# object to pack
obj = {'list': [True, False, None, 1, 'abc'], 'str': 'blah', 'bytes': b'012345'}

msgpack.pack(obj, buf)
print("packed   [{}]: {}".format(len(buf.getvalue()), buf.getvalue()))

# unpack
buf.seek(0)
obj_unpacked = msgpack.unpack(buf)
print("unpacked [{}]: {}".format(len(repr(obj)), obj_unpacked))


----- wifi-server

packed   [38]: b'\x83\xa3str\xa4blah\xa5bytes\xc4\x06012345\xa4list\x95\xc3\xc2\xc0\x01\xa3abc'
unpacked [74]: {'bytes': b'012345', 'list': [True, False, None, 1, 'abc'], 'str': 'blah'}

----- wifi-client

packed   [38]: b'\x83\xa3str\xa4blah\xa5bytes\xc4\x06012345\xa4list\x95\xc3\xc2\xc0\x01\xa3abc'
unpacked [74]: {'bytes': b'012345', 'list': [True, False, None, 1, 'abc'], 'str': 'blah'}

A more sophisticated example that makes use of MessagePack’s custom types to serialize MyClass:

%%connect wifi-server wifi-client

from msgpack import pack, unpack, ExtType
from io import BytesIO

class MyClass:
    def __init__(self, val):
        self.value = val
    def __str__(self):
        return str(self.value)

def encoder(obj):
    if isinstance(obj, MyClass):
        # represent MyClass as object with code == 1
        # valid codes are in the range 0~127
        return ExtType(1, obj.value)
    return "no encoder for {}".format(obj)

def decoder(code, data):
    if code == 1:
        # code == 1 is our code for MyClass
        return MyClass(data)
    return "no decoder for type {}".format(code)

data = MyClass(b'my_value')

buffer = BytesIO()
pack(data, buffer, default=encoder)
buffer.seek(0)
decoded = unpack(buffer, ext_hook=decoder)
print("{} -> {} -> {}".format(data, buffer.getvalue(), decoded))


----- wifi-server

b'my_value' -> b'\xd7\x01my_value' -> b'my_value'

----- wifi-client

b'my_value' -> b'\xd7\x01my_value' -> b'my_value'

RPC uses the encoder on the server and the decoder on the client to send “proxies” of classes (e.g. instances of socket) from the esp32 to the client.

FinaliserProxy#

CPython calls __del__ when the garbage collector sweeps an object. MicroPython implements this feature only for classes written in C.

FinaliserProxy is a special class written in C that calls __del__ of derived classes.

%%connect wifi-server wifi-client

try:
    from finaliserproxy import FinaliserProxy
except ImportError:
    # CPython compatibility (and ports that do not implement this feature)
    print("no finaliser proxy ...")
    class FinaliserProxy:
        def __init__(self, cb):
            pass
    
import gc

class FP(FinaliserProxy):
    def __init__(self, desc):
        self.desc = desc
        super().__init__(self.__del__)
    def __del__(self):
        print("__del__: finalise", self.desc)

for i in range(20):
    f = FP("obj {}".format(i))
    if i % 4 == 0:
        print("--------- collect ...", i)
        gc.collect()

gc.collect()

print("DONE")


----- wifi-server

--------- collect ... 0
__del__: finalise obj 19
--------- collect ... 4
__del__: finalise obj 1
__del__: finalise obj 2
__del__: finalise obj 3
__del__: finalise obj 0
--------- collect ... 8
__del__: finalise obj 5
__del__: finalise obj 6
__del__: finalise obj 7
__del__: finalise obj 4
--------- collect ... 12
__del__: finalise obj 9
__del__: finalise obj 10
__del__: finalise obj 11
__del__: finalise obj 8
--------- collect ... 16
__del__: finalise obj 13
__del__: finalise obj 14
__del__: finalise obj 15
__del__: finalise obj 12
__del__: finalise obj 17
__del__: finalise obj 18
__del__: finalise obj 16
DONE

----- wifi-client

--------- collect ... 0
--------- collect ... 4
__del__: finalise obj 1
__del__: finalise obj 2
__del__: finalise obj 3
__del__: finalise obj 0
--------- collect ... 8
__del__: finalise obj 5
__del__: finalise obj 6
__del__: finalise obj 7
__del__: finalise obj 4
--------- collect ... 12
__del__: finalise obj 9
__del__: finalise obj 10
__del__: finalise obj 11
__del__: finalise obj 8
--------- collect ... 16
__del__: finalise obj 13
__del__: finalise obj 14
__del__: finalise obj 15
__del__: finalise obj 12
__del__: finalise obj 17
__del__: finalise obj 18
__del__: finalise obj 16
DONE

RPC uses this feature on the wifi client to advise the server when a proxy object has been collected. The server in turn makes the actual object available for garbage collection.