{
"cells": [
{
"cell_type": "markdown",
"id": "c4f7281f-abdf-41c3-85c1-591720a7d8d1",
"metadata": {},
"source": [
"# Sockets\n",
"\n",
"Once an internet connection has been established, MicroPython uses [sockets](https://docs.micropython.org/en/latest/library/usocket.html) to access resources on the network, just like CPython (and pretty much [all programming languages](https://en.wikipedia.org/wiki/Network_socket)).\n",
"\n",
"Sockets are quite low level; frequently higher level libraries can be used instead. But if you want to write your own webserver, for example, you likely will use sockets.\n",
"\n",
"Examples presented are adapted from the [MicroPython github repository](https://github.com/micropython/micropython/tree/master/examples/network). Check them out for additional information.\n",
"\n",
"## http Client\n",
"\n",
"The code below first looks up the ip address of the server (`google.com`). It then creates a `socket`, connects to it at port 80 and downloads 2000 bytes. "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0df7eb05-87f8-44d7-9ef7-a4cb7523e89d",
"metadata": {
"tags": [
"hide-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[46m\u001b[30mConnected to esp32 @ serial:///dev/ttyUSB0\u001b[0m\n",
"Address information: [(2, 1, 0, 'google.com', ('142.250.191.46', 80))]\n",
"\n",
"Response:\n",
"HTTP/1.0 200 OK\n",
"Date: Thu, 09 Dec 2021 20:56:17 GMT\n",
"Expires: -1\n",
"Cache-Control: private, max-age=0\n",
"Content-Type: text/html; charset=ISO-8859-1\n",
"P3P: CP=\"This is not a P3P policy! See g.co/p3phelp for more info.\"\n",
"Server: gws\n",
"X-XSS-Protection: 0\n",
"X-Frame-Options: SAMEORIGIN\n",
"Set-Cookie: 1P_JAR=2021-12-09-20; expires=Sat, 08-Jan-2022 20:56:17 GMT; path=/; domain=.google.com; Secure\n",
"Set-Cookie: NID=511=EtAZ6YCY123hQF0O5EzPwAwG_dy5oAVQD0ph_rSNLdxsxROCEABHQiSqm4eQexkyIejMn_-9NJ6LgOr5Y-Zv_bwnqXZ47UojX7-TWcK2gcqBTf9tyiD7nsN8BoH6vA2VJPCBOfZPF3uJXN-dnd4Do35y8F6L9jri2ASzEYLoWPE; expires=Fri, 10-Jun-2022 20:56:17 GMT; path=/; domain=.google.com; HttpOnly\n",
"Accept-Ranges: none\n",
"Vary: Accept-Encoding\n",
"\n",
"cert.conf\n",
"[req]\n",
"distinguished_name = req_distinguished_name\n",
"x509_extensions = v3_req\n",
"prompt = no\n",
"[req_distinguished_name]\n",
"C = US\n",
"ST = CA\n",
"L = San Francisco\n",
"O = MicroPython Webserver\n",
"OU = iot49-$my_ip\n",
"CN = iot49-$my_ip\n",
"[v3_req]\n",
"keyUsage = critical, digitalSignature, keyAgreement\n",
"extendedKeyUsage = serverAuth\n",
"subjectAltName = @alt_names\n",
"[alt_names]\n",
"DNS.1 = $my_ip\n",
"IP.1 = $my_ip\n",
"EOF\n",
"\n",
"# create the certificate and private key\n",
"openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \\\n",
" -keyout cert.key -outform PEM -out cert.crt -config cert.conf"
]
},
{
"cell_type": "markdown",
"id": "6e45a02f-6dbd-4e74-9e45-c1e2b08c59a3",
"metadata": {},
"source": [
"The certificate and private key are now in folder `ssl`:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2fc3c3d2-104d-4b92-87b9-fd60aa1c18b5",
"metadata": {
"collapsed": true,
"jupyter": {
"outputs_hidden": true
},
"tags": [
"hide-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"-----BEGIN PRIVATE KEY-----\n",
"MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCyN3mOpUs6cPjJ\n",
"lK1OjpCTBQKjcHlnykxzg/xy8L34y1l0RQ3RKaOv0+ZvSylENCARvz3NhxyX0tJv\n",
"UbgjPEFAeWlOME9nzOW3P/mu0xWRKnq+XvbEa7E5Dc+xNgW7XmCyDNmgrjIh8C8W\n",
"1zr5Shw1hL/S4NyMWn7ZiT1Ms/jIyLkQcT+Jq9he45R+7tvQGv064psmUqWwHZUB\n",
"773N3tzpZQQcb9jukwU4bcfQy01heoJjwENwOgiw2dEiASSQ0CwkOvOXc4NLdLgI\n",
"VSf1SPF0pnw6215vygXEkcvtcOfqYdDh9G7CA36xnIYiB0usvX5WFpJKh8rMXh9W\n",
"g5ZB69OlAgMBAAECggEADV+oWZIB4TLVGJt1ne2I39+CYS1xjt5ZelmvOhjiyKbJ\n",
"4bXE4atUQz+NjVCAmkOnHG3Tf3biKGqDrCLfxJUi+GmkA2AQtWNk4amFXR8uASTE\n",
"7UBHGFpUhEmLCmtZZsQgUXwxYKNx06YognhITqjHESUTGAoTwtyPpFduKDFhO//k\n",
"BIAn7/pGdgIBKLRut3PamAkLrx1ByAmA9q5im8UBWmpWiHpVmVXlZvxnhs5Ed3au\n",
"WyRrVl8LC31s3BxZ3VPVV8gIOpc28s81pGucm0cIX0NJCCIdZsdzxpT7ypYm+DVg\n",
"T2LN8EAhwU40So3b1zz7bjQJPOTfNk0hWMCZWx+5wQKBgQDWlTPRvvsMJMjavFer\n",
"i+2l+DF56B94rtCMotAfaofE6xnxRzXrS3setKftkCSaMNm5uEMmNhYlEYTXT9Od\n",
"2KF3GMlRKbghPsJcMwnUeGlyp2QdCmd+12ZUDNFfEODzK5ZFYJotkWtllfPwu96e\n",
"jiNik3jfHY8TUQpdzw7VVinOOQKBgQDUnWDNdZ+zie42BwKoTt7XbCU5YzaQkYbd\n",
"QvG+jJY6e6eTXO/naGcxHSIQo0BmehFv+sc2sTgdymhs/OcmpSWCx+kmMjYmSmyV\n",
"UdhxN1kcZ6sWqB3NBIM1uWC7LDUX6jCbL5H8vILBeptHYhdFDya5ATZZsvO2y8jK\n",
"fQxUCf8wzQKBgQCrKqcEN5hgDnOdb8FrGJo/2uP0f0Gjbabzl+f2N28HmBXAjfIn\n",
"t7UFQEv3xxQ7Xp4+dAo0T86IURoq+gUukx/xNXdY47N56Wr2SswbjNVoXLgSJjt6\n",
"RW3du7/DWl5l+q3Kt40kriwCA4Rr0iB5T55QQpyXNSfs26cuPz1w1WNRmQKBgFdS\n",
"8CBS0C3oV7s+89t20VW/KCbC1fVYoACebzWo/ka05OXEhRARNFjas1QMCPZN6n2I\n",
"jWusK/UoXe1tje60Y4ysWNkERHNDnAdUH0aYyfO9rGpY0CyVTuKw6cbWaFQTLrV+\n",
"O0KHlliq573QzufhSjEwC0eaFTkdx7FK3NZjaLl9AoGBAMGoJuMOGgycHuueeZ5V\n",
"0KucO3lIevLG/9vEM2fG46zC3IYzny21N8b8H90Z4p+gnxN1hx1yhHZyvyKtH7A4\n",
"1zRzBd+X/gU1PJSwv0f5CMXuwDkaWaAwYzs8bzh6sOQKEDdU5c4IWOJsoI3s9WOV\n",
"QQQwn30oLuq504yT+M2AGDw3\n",
"-----END PRIVATE KEY-----\n",
"-----BEGIN CERTIFICATE-----\n",
"MIIDuDCCAqCgAwIBAgIUIf8Yr/YcTumB6OPUJ8TAY/qL3BwwDQYJKoZIhvcNAQEL\n",
"BQAwcjELMAkGA1UEBhMCVVMxCzAJBgNVBAgMAkNBMRYwFAYDVQQHDA1TYW4gRnJh\n",
"bmNpc2NvMR4wHAYDVQQKDBVNaWNyb1B5dGhvbiBXZWJzZXJ2ZXIxDjAMBgNVBAsM\n",
"BWlvdDQ5MQ4wDAYDVQQDDAVpb3Q0OTAeFw0yMTEyMTMwMjE3NTVaFw0zMTEyMTEw\n",
"MjE3NTVaMHIxCzAJBgNVBAYTAlVTMQswCQYDVQQIDAJDQTEWMBQGA1UEBwwNU2Fu\n",
"IEZyYW5jaXNjbzEeMBwGA1UECgwVTWljcm9QeXRob24gV2Vic2VydmVyMQ4wDAYD\n",
"VQQLDAVpb3Q0OTEOMAwGA1UEAwwFaW90NDkwggEiMA0GCSqGSIb3DQEBAQUAA4IB\n",
"DwAwggEKAoIBAQCyN3mOpUs6cPjJlK1OjpCTBQKjcHlnykxzg/xy8L34y1l0RQ3R\n",
"KaOv0+ZvSylENCARvz3NhxyX0tJvUbgjPEFAeWlOME9nzOW3P/mu0xWRKnq+XvbE\n",
"a7E5Dc+xNgW7XmCyDNmgrjIh8C8W1zr5Shw1hL/S4NyMWn7ZiT1Ms/jIyLkQcT+J\n",
"q9he45R+7tvQGv064psmUqWwHZUB773N3tzpZQQcb9jukwU4bcfQy01heoJjwENw\n",
"Ogiw2dEiASSQ0CwkOvOXc4NLdLgIVSf1SPF0pnw6215vygXEkcvtcOfqYdDh9G7C\n",
"A36xnIYiB0usvX5WFpJKh8rMXh9Wg5ZB69OlAgMBAAGjRjBEMA4GA1UdDwEB/wQE\n",
"AwIDiDATBgNVHSUEDDAKBggrBgEFBQcDATAdBgNVHREEFjAUggwxMC4zOS40MC4x\n",
"NjiHBAonKKgwDQYJKoZIhvcNAQELBQADggEBAIep2yjLYlLOUannmn43ogN66w6J\n",
"Pi1qDyrElK7SwHf5tHZ/rLrfIU5bVIgs98VylqtVmiG4bqDrB9kXZupoQkseiPk0\n",
"CWjWQ+Kz3FYvhnA8ylmNU0OL15TK2igc5IMBsT8b/zr4FCJ2qNQXOEsz3vH1bLBe\n",
"E9nw47/usz6JeGJroNR14WMqupLeJpga9Zf7tmpsqtx72uAJv4HUWbO1Mo0EcgcB\n",
"SSlSci/SaK6eCiLoMBKXpnVN2bR4GulJ+Anh3uiFK+bhGYsf+XrnQ58E7P3wpVcm\n",
"AQRWcEIRLfJkqnskyXLjZ0wGMULb/cZgbeNnRr3iVNGyGEw4ezqAvbd9g8U=\n",
"-----END CERTIFICATE-----\n"
]
}
],
"source": [
"!cat $IOT_PROJECTS/internet/ssl/cert.key\n",
"!cat $IOT_PROJECTS/internet/ssl/cert.crt"
]
},
{
"cell_type": "markdown",
"id": "b31e2e0b-14d9-4af1-a7b6-769e5b04aa5f",
"metadata": {
"tags": [
"hide-output"
]
},
"source": [
"Let's copy them to the ESP32, without the first and last lines:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "98a208d3-50c2-4c2c-8437-74497f087e7e",
"metadata": {},
"outputs": [],
"source": [
"%%bash\n",
"\n",
"cd $IOT_PROJECTS/internet\n",
"\n",
"mkdir -p code/ssl\n",
"\n",
"sed '1d; $d' ssl/cert.key > code/ssl/cert.key\n",
"sed '1d; $d' ssl/cert.crt > code/ssl/cert.crt"
]
},
{
"cell_type": "markdown",
"id": "4fda3932-f40c-44c9-ba80-220a36e37c5e",
"metadata": {},
"source": [
"Verify that the certificate works with CPython https server:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "1ef02f7a-8b38-4083-af73-bea4a3b2b8d0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"serving at https://10.39.40.200:4443\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"10.39.40.114 - - [12/Dec/2021 18:13:11] \"GET / HTTP/1.1\" 200 -\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"so long ...\n"
]
}
],
"source": [
"%%host\n",
"\n",
"import http.server, ssl, socket\n",
"\n",
"PORT = 4443\n",
"\n",
"with http.server.HTTPServer(('0.0.0.0', PORT), http.server.SimpleHTTPRequestHandler) as httpd:\n",
" httpd.socket = ssl.wrap_socket(httpd.socket,\n",
" server_side=True,\n",
" keyfile='ssl/cert.key',\n",
" certfile='ssl/cert.crt',\n",
" ssl_version=ssl.PROTOCOL_TLS)\n",
"\n",
" with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as s:\n",
" s.connect(('0.0.0.1', 80))\n",
" my_ip = s.getsockname()[0]\n",
"\n",
" print(f\"serving at https://{my_ip}:{PORT}\")\n",
"\n",
" try:\n",
" httpd.serve_forever()\n",
" except KeyboardInterrupt:\n",
" print(\"so long ...\")"
]
},
{
"cell_type": "markdown",
"id": "098c5f94-10f6-4460-9464-0c5d45243f31",
"metadata": {},
"source": [
"Finally run the server with code adapted from a [micropython example](https://github.com/micropython/micropython/blob/master/examples/network/http_server_ssl.py)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "b3352a8b-129c-46a4-b778-17dc1528811b",
"metadata": {
"tags": [
"hide-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32mDirectories match\n",
"\u001b[0m\n",
"\u001b[46m\u001b[31m!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\u001b[0m\n",
"\u001b[46m\u001b[31m!!!!! softreset ... !!!!!\u001b[0m\n",
"\u001b[46m\u001b[31m!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!\u001b[0m\n",
"Listening at https://10.39.40.168:8443/\n",
"OSError (-30592, 'MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE')\n",
"OSError (-30592, 'MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE')\n",
"\n",
"----- Request 1\n",
"GET / HTTP/1.1\n",
"Host: 10.39.40.168:8443\n",
"Connection: keep-alive\n",
"Cache-Control: max-age=0\n",
"sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"\n",
"sec-ch-ua-mobile: ?0\n",
"sec-ch-ua-platform: \"macOS\"\n",
"DNT: 1\n",
"Upgrade-Insecure-Requests: 1\n",
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36\n",
"Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9\n",
"Sec-Fetch-Site: cross-site\n",
"Sec-Fetch-Mode: navigate\n",
"Sec-Fetch-User: ?1\n",
"Sec-Fetch-Dest: document\n",
"Referer: https://server.local/\n",
"Accept-Encoding: gzip, deflate, br\n",
"Accept-Language: en-US,en;q=0.9,de-CH;q=0.8,de;q=0.7,fr-FR;q=0.6,fr;q=0.5,zh-HK;q=0.4,zh-CN;q=0.3,zh-TW;q=0.2,zh;q=0.1\n",
"\n",
"OSError (-30592, 'MBEDTLS_ERR_SSL_FATAL_ALERT_MESSAGE')\n",
"\n",
"----- Request 2\n",
"GET /favicon.ico HTTP/1.1\n",
"Host: 10.39.40.168:8443\n",
"Connection: keep-alive\n",
"sec-ch-ua: \" Not A;Brand\";v=\"99\", \"Chromium\";v=\"96\", \"Google Chrome\";v=\"96\"\n",
"DNT: 1\n",
"sec-ch-ua-mobile: ?0\n",
"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.55 Safari/537.36\n",
"sec-ch-ua-platform: \"macOS\"\n",
"Accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8\n",
"Sec-Fetch-Site: same-origin\n",
"Sec-Fetch-Mode: no-cors\n",
"Sec-Fetch-Dest: image\n",
"Referer: https://10.39.40.168:8443/\n",
"Accept-Encoding: gzip, deflate, br\n",
"Accept-Language: en-US,en;q=0.9,de-CH;q=0.8,de;q=0.7,fr-FR;q=0.6,fr;q=0.5,zh-HK;q=0.4,zh-CN;q=0.3,zh-TW;q=0.2,zh;q=0.1\n",
"\n",
"\n",
"----- Request 3\n",
"GET / HTTP/1.1\n",
"Host: 10.39.40.168:8443\n",
"User-Agent: curl/7.68.0\n",
"Accept: */*\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Interrupted\u001b[0m\n"
]
}
],
"source": [
"%rsync\n",
"%softreset\n",
"\n",
"import ubinascii as binascii\n",
"import network\n",
"\n",
"try:\n",
" import usocket as socket\n",
"except:\n",
" import socket\n",
"import ussl as ssl\n",
"\n",
"# Read the certificate and key, convert to binary\n",
"\n",
"with open('/ssl/cert.key') as f:\n",
" key = binascii.a2b_base64(f.read())\n",
"\n",
"with open('/ssl/cert.crt') as f:\n",
" cert = binascii.a2b_base64(f.read())\n",
"\n",
"\n",
"CONTENT = b\"\"\"\\\n",
"HTTP/1.0 200 OK\n",
"\n",
"Hello #%d from MicroPython!\n",
"\"\"\"\n",
"\n",
"def main():\n",
" s = socket.socket()\n",
"\n",
" # Binding to all interfaces - server will be accessible to other hosts!\n",
" ai = socket.getaddrinfo(\"0.0.0.0\", 8443)\n",
" addr = ai[0][-1]\n",
"\n",
" s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)\n",
" s.bind(addr)\n",
" s.listen(5)\n",
"\n",
" ip = network.WLAN(network.STA_IF).ifconfig()[0]\n",
" print(\"Listening at https://{}:8443/\".format(ip))\n",
"\n",
" counter = 1\n",
" while True:\n",
" client_s, client_addr = s.accept()\n",
" # CPython uses key keyfile/certfile arguments, but MicroPython uses key/cert\n",
" try:\n",
" client_s = ssl.wrap_socket(client_s, server_side=True, key=key, cert=cert)\n",
" print(\"\\n----- Request\", counter)\n",
" # Both CPython and MicroPython SSLSocket objects support read() and\n",
" # write() methods.\n",
" # Browsers are prone to terminate SSL connection abruptly if they\n",
" # see unknown certificate, etc. We must continue in such case -\n",
" # next request they issue will likely be more well-behaving and\n",
" # will succeed.\n",
" req = client_s.readline()\n",
" print(req.decode(), end=\"\")\n",
" while True:\n",
" h = client_s.readline()\n",
" if h == b\"\" or h == b\"\\r\\n\":\n",
" break\n",
" print(h.decode(), end=\"\")\n",
" if req:\n",
" client_s.write(CONTENT % counter)\n",
" counter += 1\n",
" print()\n",
" except OSError as e:\n",
" print(\"OSError\", e)\n",
" finally:\n",
" client_s.close() \n",
"\n",
"main()"
]
},
{
"cell_type": "markdown",
"id": "41f32247-a1d7-44bd-934a-7e2d9a97db0a",
"metadata": {},
"source": [
"You'll get a warning that the connection is not private. That's because of the self-signed certificate - the browser does not know us and hence can ascertain that the certificate is legitimate. If you add it to the browser, the error disappears.\n",
"\n",
"Running (substitute the correct IP address)\n",
"\n",
"```bash\n",
"curl --insecure -v https://10.39.40.168:8443/\n",
"```\n",
"\n",
"from the command line prints information about the certificate (and avoids the OSError). The `--insecure` flag is needed since `curl` doesn't trust us in our capacity as \"Certificate Authority\"."
]
},
{
"cell_type": "markdown",
"id": "77a175be-52e6-49da-a637-87e43d1b004f",
"metadata": {},
"source": [
"## Wireshark\n",
"\n",
"But how do we know that the transmitted data is indeed encrypted and secure? We need a spy on the internet that intercepts and views packets.\n",
"\n",
"[Wireshark](https://www.wireshark.org/) is a program that does just that and it's preinstalled in *ide49*. Go to the home screen and click on the blue shark fin. \n",
"\n",
"Highlight the wlan (`wlp3s0`) and enter `port 8080` as the capture filter. Then press the shark fin in the toolbar to start the capture. For a first test, start the unencrypted http server and run `curl http://10.39.40.168:8080/` (update the ip address). Wireshark will list all the captured packets and you can read `Hello from MicroPython` in the clear. Anyone with access to the network can read your message.\n",
"\n",
"\n",
"\n",
"Now stop Wireshark (press first the red square and then the tool with the cross) and change the capture filter `port 8443` and start the capture. Start the https server and run `curl --insecure -v https://10.39.40.168:8443/`. Now all packets are encrypted:\n",
"\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9577ded4-e461-410e-8cad-ac3c83743171",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "IoT",
"language": "python",
"name": "iot_kernel"
},
"language_info": {
"codemirror_mode": {
"name": "python",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"pygments_lexer": "python3",
"version": "3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}