# Overview
*ide49* is based on [Balena](https://www.balena.io/what-is-balena), a version of [Docker](https://www.docker.com/) optimized for IoT devices, rather than datacenters.
```{figure} figures/app.png
:alt: balena_app
:width: 400px
:align: center
Structure of a Docker application. A single Linux kernel is shared between the host and one or more user containers.
```
The balenaOS takes the function of the host operating system. Unlike other operating systems (Windows, MacOS, ...), it is highly specialized to run individual services in so-called containers. In *ide49* available containers include micropython, code-server, duplicati, etc and more can be added over time to provide additional functionality.
A set of [configuration files](https://github.com/iot49/ide49) describes the application. The [docker-compose.yml](https://github.com/iot49/ide49/blob/main/docker-compose.yml) file lists all user containers along with configuration parameters and information about how they interact. E.g. containers may be permitted to share files (`volumes`) or communicate over dedicated networks that are inaccessible from outside.
Each container presents a separate instance of Linux, and usually implement just one function. This arrangement minimizes undesired interaction e.g. between incompatible libraries used in various application programs. It's like having a separate computer for each task (code editor, backup, etc).
The software installed in each container is described with a Dockerfile. For example, the first line in the Dockerfile for the [micropython](https://github.com/iot49/ide49/blob/main/core/micropython/Dockerfile) specifies that this service is derived from a slightly customized version of an "official" [Jupyter docker image](https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html#jupyter-scipy-notebook). This image is based on Ubuntu Linux and includes, notably, the Jupyter Notebook server as well as a host of libraries for scientific computation and plotting. Following the FROM statement are instructions for installing additional software and configuration files including the IoT Kernel used for interacting with the MicroPython REPL.
## BalenaEngine
In Balena, the balenaEngine takes the function of the docker deamon in standard docker apps. It is accessed from the host with the `balena-engine` command.
balena-engine –help
To get a list of all running containers, type:
%%service host
balena-engine ps --format "table {{.Names}}\t{{.Size}}\t{{.Image}}"
NAMES SIZE IMAGE
micropython_4326350_2006136_52d3260cfa7974104feb37cd5f7b10e2 108MB (virtual 3.31GB) 92e31d0562f3
esp-idf_4326361_2006136_52d3260cfa7974104feb37cd5f7b10e2 211MB (virtual 4.08GB) 3a3cec03f2f0
bluetooth_4326363_2006136_52d3260cfa7974104feb37cd5f7b10e2 140MB (virtual 917MB) b8f19d396651
rust_4326364_2006136_52d3260cfa7974104feb37cd5f7b10e2 106MB (virtual 1.09GB) d731539ad951
arm32_4326358_2006136_52d3260cfa7974104feb37cd5f7b10e2 211MB (virtual 2.14GB) 8d13346713a1
nginx_4326349_2006136_52d3260cfa7974104feb37cd5f7b10e2 3B (virtual 29.7MB) 9aaac23e700e
balena-cli_4326362_2006136_52d3260cfa7974104feb37cd5f7b10e2 0B (virtual 1.64GB) bb952f8265bd
wireshark_4326355_2006136_52d3260cfa7974104feb37cd5f7b10e2 1.03MB (virtual 1.21GB) be89f6dc2961
mosquitto_4326354_2006136_52d3260cfa7974104feb37cd5f7b10e2 0B (virtual 11.8MB) 36a40df3c01e
plotserver_4326356_2006136_52d3260cfa7974104feb37cd5f7b10e2 0B (virtual 617MB) 5e7daca5ada6
smb_4326353_2006136_52d3260cfa7974104feb37cd5f7b10e2 353kB (virtual 265MB) 0d9849478188
duplicati_4326352_2006136_52d3260cfa7974104feb37cd5f7b10e2 331kB (virtual 659MB) 0910e1cc6298
code-server_4326351_2006136_52d3260cfa7974104feb37cd5f7b10e2 336kB (virtual 640MB) abaf7dc162b3
balena_supervisor 45B (virtual 67.1MB) registry2.balena-cloud.com/v2/83f98d1c7daeec48d6f34806ab7a6fc0:latest
The balenaEngine has many more features. For example, to get the build history for the arm32 image run
%%service host
balena-engine history 92e31d0562f3 --format "table {{.Size}}\t{{.CreatedBy}}" # --no-trunc
Show code cell output
SIZE CREATED BY
22.7MB /bin/bash -o pipefail -c echo "force rebuild…
9.49MB /bin/bash -o pipefail -c pip install --defau…
355MB /bin/bash -o pipefail -c mamba install --qui…
0B /bin/bash -o pipefail -c #(nop) USER 1000
3.52kB /bin/bash -o pipefail -c groupadd gpio && u…
527B /bin/bash -o pipefail -c #(nop) COPY file:bb…
501B /bin/bash -o pipefail -c #(nop) COPY file:f3…
119B /bin/bash -o pipefail -c #(nop) COPY file:54…
182MB /bin/bash -o pipefail -c apt-get update --ye…
0B /bin/bash -o pipefail -c #(nop) ENV DEBIAN_…
0B /bin/bash -o pipefail -c #(nop) USER root
0B CMD ["/bin/bash" "/usr/local/bin/start.sh"]
0B WORKDIR /home/iot
0B USER 1000
11.5kB RUN |1 IOT_USER=iot /bin/bash -o pipefail -c…
9.85kB COPY conf /usr/local/bin/ # buildkit
0B ENV HOME=/home/iot
0B ENV NB_USER=iot
0B ARG IOT_USER=iot
0B ENV UDEV=on
35.2MB RUN /bin/bash -o pipefail -c chmod u+xs /usr…
18.3kB COPY /tmp/add_hostname /usr/local/bin # buil…
0B USER root
0B LABEL description=Run container - jupyter st…
0B WORKDIR /home/jovyan
0B USER 1000
86.2kB RUN /bin/bash -o pipefail -c MPLBACKEND=Agg …
0B ENV XDG_CACHE_HOME=/home/jovyan/.cache/
2.31MB RUN /bin/bash -o pipefail -c git clone https…
0B WORKDIR /tmp
861MB RUN /bin/bash -o pipefail -c mamba install -…
0B USER 1000
359MB RUN /bin/bash -o pipefail -c apt-get update …
0B USER root
0B LABEL maintainer=Jupyter Project <jupyter@go…
0B USER 1000
9.99kB RUN /bin/bash -o pipefail -c update-alternat…
697MB RUN /bin/bash -o pipefail -c apt-get update …
0B USER root
0B LABEL maintainer=Jupyter Project <jupyter@go…
0B WORKDIR /home/jovyan
0B USER 1000
3.66kB RUN |5 NB_USER=jovyan NB_UID=1000 NB_GID=100…
0B USER root
1.84kB COPY jupyter_notebook_config.py /etc/jupyter…
11kB COPY start.sh start-notebook.sh start-single…
0B CMD ["start-notebook.sh"]
0B ENTRYPOINT ["tini" "-g" "--"]
0B EXPOSE map[8888/tcp:{}]
326MB RUN |5 NB_USER=jovyan NB_UID=1000 NB_GID=100…
251MB RUN |5 NB_USER=jovyan NB_UID=1000 NB_GID=100…
0B ARG CONDA_MIRROR=https://github.com/conda-fo…
0B WORKDIR /tmp
0B RUN |4 NB_USER=jovyan NB_UID=1000 NB_GID=100…
0B ARG PYTHON_VERSION=default
0B USER 1000
11.6kB RUN |3 NB_USER=jovyan NB_UID=1000 NB_GID=100…
3.82kB RUN |3 NB_USER=jovyan NB_UID=1000 NB_GID=100…
0B RUN |3 NB_USER=jovyan NB_UID=1000 NB_GID=100…
1.03kB COPY fix-permissions /usr/local/bin/fix-perm…
0B ENV PATH=/opt/conda/bin:/usr/local/sbin:/usr…
0B ENV CONDA_DIR=/opt/conda SHELL=/bin/bash NB_…
30.8MB RUN |3 NB_USER=jovyan NB_UID=1000 NB_GID=100…
0B ENV DEBIAN_FRONTEND=noninteractive
0B USER root
0B SHELL [/bin/bash -o pipefail -c]
0B ARG NB_GID=100
0B ARG NB_UID=1000
0B ARG NB_USER=jovyan
0B LABEL maintainer=Jupyter Project <jupyter@go…
0B /bin/sh -c #(nop) CMD ["bash"]
72.8MB /bin/sh -c #(nop) ADD file:5d68d27cc15a80653…
Networking#
Containers use networks for communication. ide49 makes use of two separate networks:
an internal network used only for communication within the app, and
the host network, used to access the Internet.
Most containers have access only to the internal network. The nginx webserver acts as a reverse proxy to pass requests received from browsers on port 443 (and port 80, which is forwarded to 443) to the appropriate container on the internal network. The single ingress allows centralized handling of encryption and password verification in one place. The nginx configration is at
!cat /service-config/nginx/nginx.conf
Show code cell output
# /etc/nginx/nginx.conf
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 256;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# gzip on;
# http -> https redirect
server {
listen 80;
return 301 https://$host$request_uri;
}
server {
# disable port 80 after enabling https redirect (above)
# listen 80;
# Enable ssl/tls (https): copy certificate to /etc/nginx/ssl and uncomment lines below
listen 443 ssl;
ssl_certificate /etc/nginx/ssl/cert.crt;
ssl_certificate_key /etc/nginx/ssl/cert.key;
# password authentication (default: iot49/iot49)
auth_basic "iot49: Electronics for IoT";
auth_basic_user_file htpasswd;
# Docker DNS
resolver 127.0.0.11;
# MicroPython Development (Jupyter)
location /micropython/ {
# proxy for network_mode: host
# proxy_pass http://172.17.0.1:8888;
proxy_pass http://micropython:8888;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# Code-server
location /code-server/ {
proxy_pass http://code-server:8443/;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# Duplicati
location /duplicati {
return 301 $scheme://$host/duplicati/;
}
location ^~ /duplicati/ {
rewrite /duplicati(.*) $1 break;
proxy_pass http://duplicati:8200;
}
# Wireshark
# Don't know how to proxy, so using redirect to http instead ...
# https://serverfault.com/questions/932628/how-to-handle-relative-urls-correctly-with-a-nginx-reverse-proxy
location /wireshark {
return 301 http://$http_host:3000/;
}
# Plotserver
location /plotserver/ {
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_pass http://plotserver:8080/;
}
include plotserver.conf;
# ARM32 Cross Compiler (Jupyter)
location /arm32/ {
proxy_pass http://arm32:8889;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# ESP32 Cross Compiler (Jupyter)
location /esp-idf/ {
proxy_pass http://esp-idf:8890;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# Rust (Jupyter)
location /rust/ {
proxy_pass http://rust:8893;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# Balena-CLI
location /balena-cli/ {
proxy_pass http://balena-cli:8891;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# Bluetooth
location /bluetooth/ {
# proxy for network_mode: host
proxy_pass http://172.17.0.1:8892;
proxy_redirect off;
proxy_set_header Host $host;
# websocket support
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_read_timeout 86400;
}
# static content
location / {
auth_basic off;
root /etc/nginx/html;
index index.html index.htm;
}
}
}
The network confiuration is available from the balena-engine:
%%service host
balena-engine ps --format "table {{.Names}}\t{{.Ports}}"
Show code cell output
NAMES PORTS
micropython_4326350_2006136_52d3260cfa7974104feb37cd5f7b10e2 8888/tcp
esp-idf_4326361_2006136_52d3260cfa7974104feb37cd5f7b10e2 8888/tcp, 8890/tcp
bluetooth_4326363_2006136_52d3260cfa7974104feb37cd5f7b10e2
rust_4326364_2006136_52d3260cfa7974104feb37cd5f7b10e2 8888/tcp, 8893/tcp
arm32_4326358_2006136_52d3260cfa7974104feb37cd5f7b10e2 8888-8889/tcp
nginx_4326349_2006136_52d3260cfa7974104feb37cd5f7b10e2 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp
balena-cli_4326362_2006136_52d3260cfa7974104feb37cd5f7b10e2 8891/tcp
wireshark_4326355_2006136_52d3260cfa7974104feb37cd5f7b10e2
mosquitto_4326354_2006136_52d3260cfa7974104feb37cd5f7b10e2 0.0.0.0:1883->1883/tcp, 0.0.0.0:8883->8883/tcp, 0.0.0.0:9001-9002->9001-9002/tcp
plotserver_4326356_2006136_52d3260cfa7974104feb37cd5f7b10e2 8080/tcp
smb_4326353_2006136_52d3260cfa7974104feb37cd5f7b10e2 0.0.0.0:139->139/tcp, 0.0.0.0:445->445/tcp
duplicati_4326352_2006136_52d3260cfa7974104feb37cd5f7b10e2 8200/tcp
code-server_4326351_2006136_52d3260cfa7974104feb37cd5f7b10e2 8443/tcp
balena_supervisor
Unlike other containers, the micropython service is member of the host network to enable access to bluetooth (if available). It also means servers (e.g. webservers) running in the micropython container can be accessed from from the internet without password protection and encryption from nginx. This is convenient for development but also poses a security risk especially if the device is not behind a firewall (as e.g. provided by a home router).
Storage#
Broadly, Docker containers distinguish two kinds of data storage.
Owned by the container#
That’s the default and includes all operating system related files, installed software (compilers, etc).
Although it’s possible to make changes (e.g. with apt-get run inside the container), those changes do not persist between updates and hence should be limited to testing new features. A better solution is to instead modify the ide49 app, per instruction in the next section.
This part of storage is “private” to the container and invisible from other containers in the same app (e.g. ide49).
Mounted into the container (volumes)#
Storage can also be “mounted” from the “host”, the underlying operating system that runs the Docker app, ide49. For example, /home/iot and the subfolders of /service-config are mounted. The volumes section of the docker-compose.yml file lists all mounts.
The same volumes can be mounted into several containers (at the same or different locations). This enables data sharing. For example, all volumes are mounted in the duplicati container for backup and micropython and code-server containers for editing.
For example the configuration for nginx (webserver), is mounted in /service-config in the code-server and micropython containers for editing, and at /etc/nginx in the nginx service.
Changes to volumes persist between container updates.
Mounts#
The duplicati service automtically mounts USB storage devices at /mnt. These devices can also be manually mounted in other containers (e.g. micropython). Avoid simultaneous access of the same storage device from multiple containers to avoid data corruption.
Below is an example of mounting an attached device (e.g. USB thumb drive).
%%bash
# 1) find device, e.g. /dev/sdc1
sudo fdisk -l
# 2) create mount point
sudo mkdir -p /mnt/media
# 3) mount volume
sudo mount /dev/sdc1 /mnt/media
# 4) use data ...
ls /mnt/media
# 5) unmount
sudo umount /mnt/media