{ "cells": [ { "cell_type": "markdown", "metadata": { "collapsed": false, "tags": [ "hide-cell" ] }, "source": [ "# Neural Networks\n", "\n", "Neural networks being a very powerful class of models, especially in cases where the learning of representations from low-level information (such as pixels, audio samples or text) is key, sensAI provides many useful abstractions for dealing with this class of models, facilitating data handling, learning and evaluation.\n", "\n", "sensAI mainly provides abstractions for PyTorch, but there is also rudimentary support for TensorFlow." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:29.915658Z", "iopub.status.busy": "2024-08-13T22:11:29.915454Z", "iopub.status.idle": "2024-08-13T22:11:29.930959Z", "shell.execute_reply": "2024-08-13T22:11:29.930330Z" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "%load_ext autoreload\n", "%autoreload 2" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:29.933962Z", "iopub.status.busy": "2024-08-13T22:11:29.933575Z", "iopub.status.idle": "2024-08-13T22:11:31.040239Z", "shell.execute_reply": "2024-08-13T22:11:31.039583Z" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": [ "%%capture\n", "import sys; sys.path.extend([\"../src\", \"..\"])\n", "import sensai\n", "import pandas as pd\n", "import numpy as np\n", "from typing import *\n", "import config\n", "import warnings\n", "import functools\n", "\n", "cfg = config.get_config()\n", "warnings.filterwarnings(\"ignore\")\n", "sensai.util.logging.configure()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Image Classification\n", "\n", "As an example use case, let us solve the classification problem of classifying digits in pixel images from the MNIST dataset. Images are greyscale (no colour information) and 28x28 pixels in size." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:31.043884Z", "iopub.status.busy": "2024-08-13T22:11:31.043577Z", "iopub.status.idle": "2024-08-13T22:11:33.260745Z", "shell.execute_reply": "2024-08-13T22:11:33.260043Z" } }, "outputs": [], "source": [ "mnist_df = pd.read_csv(cfg.datafile_path(\"mnist_train.csv.zip\"))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The data frame contains one column for every pixel, each pixel being represented by an 8-bit integer (0 to 255)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:33.264505Z", "iopub.status.busy": "2024-08-13T22:11:33.264042Z", "iopub.status.idle": "2024-08-13T22:11:33.293802Z", "shell.execute_reply": "2024-08-13T22:11:33.293054Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
label1x11x21x31x41x51x61x71x81x9...28x1928x2028x2128x2228x2328x2428x2528x2628x2728x28
05000000000...0000000000
10000000000...0000000000
24000000000...0000000000
31000000000...0000000000
49000000000...0000000000
\n", "

5 rows × 785 columns

\n", "
" ], "text/plain": [ " label 1x1 1x2 1x3 1x4 1x5 1x6 1x7 1x8 1x9 ... 28x19 28x20 \\\n", "0 5 0 0 0 0 0 0 0 0 0 ... 0 0 \n", "1 0 0 0 0 0 0 0 0 0 0 ... 0 0 \n", "2 4 0 0 0 0 0 0 0 0 0 ... 0 0 \n", "3 1 0 0 0 0 0 0 0 0 0 ... 0 0 \n", "4 9 0 0 0 0 0 0 0 0 0 ... 0 0 \n", "\n", " 28x21 28x22 28x23 28x24 28x25 28x26 28x27 28x28 \n", "0 0 0 0 0 0 0 0 0 \n", "1 0 0 0 0 0 0 0 0 \n", "2 0 0 0 0 0 0 0 0 \n", "3 0 0 0 0 0 0 0 0 \n", "4 0 0 0 0 0 0 0 0 \n", "\n", "[5 rows x 785 columns]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mnist_df.head(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's create the I/O data for our experiments." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:33.297111Z", "iopub.status.busy": "2024-08-13T22:11:33.296565Z", "iopub.status.idle": "2024-08-13T22:11:33.451881Z", "shell.execute_reply": "2024-08-13T22:11:33.451234Z" } }, "outputs": [], "source": [ "mnistIoData = sensai.InputOutputData.from_data_frame(mnist_df, \"label\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now that we have the image data separated from the labels, let's write a function to restore the 2D image arrays and take a look at some of the images." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:33.455243Z", "iopub.status.busy": "2024-08-13T22:11:33.455013Z", "iopub.status.idle": "2024-08-13T22:11:33.901475Z", "shell.execute_reply": "2024-08-13T22:11:33.900537Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAC0CAYAAABG37kPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAfKElEQVR4nO3de3CU1f3H8W8SSUBINiKQEEmUtozo6EDlEgPeipEMHSkIVpyRi3erQVGkFETwRo3FYpFbra3DxRblMgID7UAxSvASLnLxWiNtUePERNBmNwYISJ7fH/yIPnsOZHd5ds9zkvdrZv84n5zd/SZ8fZLjs895khzHcQQAAAAALJZsugAAAAAAOF0sbAAAAABYj4UNAAAAAOuxsAEAAABgPRY2AAAAAKzHwgYAAACA9VjYAAAAALAeCxsAAAAA1mNhAwAAAMB6LGwAAAAAWC9uC5sFCxbIeeedJ23btpX8/HzZvn17vN4KUNB/MIn+g2n0IEyi/2BKkuM4jtcvunz5chk7dqw899xzkp+fL3PmzJGVK1dKRUWFdOnS5ZTPbWxslKqqKklPT5ekpCSvS4OlHMeRuro6ycnJkeTkU6/HT6f/ROhBqOg/mJaoHqT/oMMxECZF03/ixEH//v2d4uLipvGxY8ecnJwcp6SkpNnnVlZWOiLCg4f2UVlZGdf+owd5nOpB//Ew/Yh3D9J/PE714BjIw+Qjkv47Qzx25MgR2blzp0ydOrUpS05OlsLCQikvL1fmNzQ0SENDQ9PY+f8TSJWVlZKRkeF1ebBUKBSS3NxcSU9PP+W8aPtPhB5E8+g/mBavHqT/EAmOgTAp0v4TEfF8YXPgwAE5duyYZGVlufKsrCz5+OOPlfklJSXy2GOPKXlGRgYNDUVzp6Wj7T8RehCRo/9gmtc9SP8hGhwDYVIkH000viva1KlTJRgMNj0qKytNl4RWhh6ESfQfTKL/YBo9CC95fsamU6dOkpKSIjU1Na68pqZGsrOzlflpaWmSlpbmdRlopaLtPxF6EN6h/2Aav4NhEsdAmOb5GZvU1FTp06ePlJaWNmWNjY1SWloqBQUFXr8d4EL/wST6D6bRgzCJ/oNpnp+xERGZOHGijBs3Tvr27Sv9+/eXOXPmSH19vdxyyy3xeDvAhf6DSfQfTKMHYRL9B5PisrAZNWqU7N+/X2bMmCHV1dXSu3dv2bBhg3IxGRAP9B9Mov9gGj0Ik+g/mBSXG3SejlAoJIFAQILBILthoEki+4IeRDj6D6Ylqi/oP+hwDIRJ0fSE8V3RAAAAAOB0sbABAAAAYD0WNgAAAACsx8IGAAAAgPVY2AAAAACwHgsbAAAAANZjYQMAAADAeixsAAAAAFiPhQ0AAAAA651hugAA9tm5c6eSzZ8/3zVesmSJMmfcuHFKdu+99yrZJZdcchrVAQCA1ogzNgAAAACsx8IGAAAAgPVY2AAAAACwHgsbAAAAANZj8wBDjh07pmTBYDDm1wu/cPvgwYPKnIqKCiVbsGCBkk2aNMk1fumll5Q5bdu2VbIpU6Yo2SOPPKIWC6vs2bNHyQoLC5UsFAq5xklJScqcpUuXKtnatWuV7JtvvomiQsBbpaWlrvFNN92kzCkrK1Oy888/P241wX4zZ85UshkzZiiZ4ziu8ebNm5U5V155pWd1AS0JZ2wAAAAAWI+FDQAAAADrsbABAAAAYD0WNgAAAACsx+YBUfj888+V7MiRI0r29ttvK9mbb77pGtfW1ipzVq1aFXtxEcjNzVUy3V3fV69e7Rqnp6crc3r16qVkXMxov+3btyvZyJEjlUy30UX4ZgEZGRnKnNTUVCU7cOCAkpWXl7vGffr0iei1oNqyZYuSff3110p23XXXJaIcK+zYscM17tu3r6FKYKvFixcr2VNPPaVkKSkpSha+uZBuIxYAepyxAQAAAGA9FjYAAAAArMfCBgAAAID1uMbmJHbv3q1kgwYNUrLTualmPOk+t6u7OVj79u2VLPxmdDk5Ocqcs846S8m4OZ2/hd+0ddeuXcqc0aNHK1lVVVVM79ejRw8lmzx5spKNGjVKyQYOHOga63r3oYceiqmu1kZ3c7+9e/cqWWu9xqaxsVHJ9u3b5xrrrq8Mv4ki8EOfffaZkjU0NBioBH61bds2JXvxxReVTHed5AcffNDs68+ePVvJdH/PvfHGG0o2ZswY1zg/P7/Z9/MLztgAAAAAsB4LGwAAAADWY2EDAAAAwHosbAAAAABYj80DTuLcc89Vsk6dOilZvDcP0F2wpbtw//XXX3eNdTcvDL8YDK3LXXfd5RovW7Ysru+3c+dOJfv222+VTHdj1/AL3t9//33P6mptlixZomQDBgwwUIk/ffnll0r2/PPPu8a6Y2fPnj3jVhPs8+qrr7rGc+fOjeh5uj5av369a5yVlRV7YfCN5cuXu8YTJkxQ5uzfv1/JdBuVXHXVVUoWfrPrSZMmRVSX7vXDX+vll1+O6LX8gDM2AAAAAKzHwgYAAACA9VjYAAAAALAeCxsAAAAA1mPzgJPo2LGjkj399NNKtm7dOiX76U9/qmT33Xdfs+/Zu3dvJQu/IFFEpH379koWfhfaSC9cRMuku3A//ILUSO+crrtI8dprr1Wy8AsVdXc41v23EclmGNzlPXaNjY2mS/C122+/vdk5PXr0SEAlsMWbb76pZDfffLNrHAqFInqtX//610qm27wI/vXdd98p2Y4dO5TsjjvucI3r6+uVObrNdKZPn65kl112mZI1NDS4xjfccIMyZ+PGjUqm07dv34jm+RFnbAAAAABYj4UNAAAAAOuxsAEAAABgvagXNlu2bJGhQ4dKTk6OJCUlyZo1a1xfdxxHZsyYIV27dpV27dpJYWGh7N2716t60cq99dZb9B+Mof9gGj0Ik+g/+F3UmwfU19dLr1695NZbb5URI0YoX581a5bMnTtXlixZIt27d5fp06dLUVGRfPTRR9K2bVtPijZl+PDhSjZo0CAlS09PV7L33nvPNf7LX/6izNHdJVa3UYDORRdd5BqH3zm7pTh48GCr7b+T2bNnj5IVFhYqWfjFrElJScqcn//850r20ksvKdnmzZuV7Le//a1rrLsou3PnzkrWq1cvJQuv7e9//7syZ9euXUp2ySWXKJmX/N5/4ccZEZGampq4v6/Namtrm51zzTXXxL+QCPm9B1uDJUuWKFlVVVWzz9NtxDJ27FgvSkoY+k/117/+Vcluu+22Zp83ePBgJVu+fLmSZWRkRFRH+HMj3SggNzdXycaNGxfRc/0o6oXNkCFDZMiQIdqvOY4jc+bMkYcffliGDRsmIiJLly6VrKwsWbNmjdx4442nVy1avWuuuUZGjhyp/Rr9h3ij/2AaPQiT6D/4nafX2Ozbt0+qq6td/7c4EAhIfn6+lJeXa5/T0NAgoVDI9QBiEUv/idCD8Ab9B9P4HQyTOAbCDzxd2FRXV4uISFZWlivPyspq+lq4kpISCQQCTQ/dKTEgErH0nwg9CG/QfzCN38EwiWMg/MD4rmhTp06VYDDY9KisrDRdEloZehAm0X8wif6DafQgvBT1NTankp2dLSLHL1bt2rVrU15TUyO9e/fWPictLU3S0tK8LCOhIr2oKxAINDtHt6GA7jOpycnG16O+FEv/idjVg5988omSzZo1S8mCwaCShV+4/8Of0Qm6CwY7dOigZNdee21EmVcOHjyoZL///e+VbNmyZXGroTl+6L9//OMfSnbo0CFPXrsl0G2k8Omnnzb7vHPOOScO1XivNf4OjrcDBw4o2QsvvKBkKSkprnFmZqYy5+GHH/asLj/ywzEw3nT/hk8++aSS6TbnKS4udo1nzpypzIn0b0qd8A18IjV37lwl0230YwtP/0Lu3r27ZGdnS2lpaVMWCoVk27ZtUlBQ4OVbAQr6DybRfzCNHoRJ9B/8IOozNt9++638+9//bhrv27dP9uzZIx07dpS8vDy5//77ZebMmdKjR4+mrf5ycnK0WyUD0fr222/lv//9b9OY/kMi0X8wjR6ESfQf/C7qhc0777wjP/vZz5rGEydOFJHjH2FZvHixTJ48Werr6+XOO++U2tpaueyyy2TDhg0tdv9yJNbu3btdH3mi/5BI9B9MowdhEv0Hv0tyHMcxXcQPhUIhCQQCEgwGT+uzhn5TX1/vGg8dOlSZo7vp4YYNG5RMd1Onli6RfeGXHmxoaFCyX/7yl0qmu3ml7iax4Tfv6tu3rzJHdz1Gt27dTllnPIRfR6b7vPKAAQOU7I033ohLPbb03y233KJkixcvVrKSkhIlmzJlSlTvZaMxY8Yome7meueff75rvHXrVmWO7hqKeEpUD/rl+GeC7nor3U0odTdFDr/GZvr06cqcGTNmxFybabYcA730+OOPK9mjjz6qZLrrg4qKipQs/GbX7dq1i6iOw4cPK9k///lPJQu/Jlv3PF1fPvbYYxHVYVI0PcFV6AAAAACsx8IGAAAAgPVY2AAAAACwHgsbAAAAANbz9AadOLn27du7xn/+85+VOZdccomS3XHHHUr2w13pTgi/EDz8RlAi+guw4V+7du1SMt1GATpr165VsiuvvPK0a0LL0K9fP9MlnJZQKKRk4Rut6DYF0F1wqxN+E75EbxQAM3Sb9bz//vsRPffqq692jSdMmOBJTUic2tpa13jhwoXKHN3fUbqNAtasWRNTDT+8ncoJN910k5K98847zb6WbrOhyZMnx1SXTThjAwAAAMB6LGwAAAAAWI+FDQAAAADrsbABAAAAYD02DzDkxz/+sZLp7hCuu5P40qVLm83q6+uVOWPHjlWyrl27nqpMGDRx4kQlcxxHya666iols32jAN33Gcsc6H3zzTeevda7776rZI2NjUpWWlrqGn/xxRfKnCNHjijZ3/72t4heP/wu3vn5+coc3R3Cjx49qmThm7Gg5dFd3D1lypSInnv55Zcr2ZIlS1zjQCAQU10wJ/z4s3///oieN3fuXCX76quvlGzRokWusW6Tnw8//FDJ6urqlEy3iUFysvtcxejRo5U54RtZtUScsQEAAABgPRY2AAAAAKzHwgYAAACA9VjYAAAAALAemwf4yHXXXadkP/nJT5TswQcfVLJXX33VNZ46daoy57PPPlOyadOmKdk555xzyjoRH+vXr3eN9+zZo8zRXTD4i1/8Il4lGRP+feq+7969eyeoGnuEX0Avov/Z3XXXXUr25JNPxvSeus0DdBs7tGnTxjU+88wzlTkXXHCBkt16661K1qdPHyUL30QjKytLmdOtWzclO3TokJL17NlTyWC3Tz/91DUeMWJEzK/1ox/9SMl0/Qa7pKamusZdunRR5ug2BTjvvPOUTHfcjYTu76+MjAwlq6qqUrJOnTq5xkOHDo2pBttxxgYAAACA9VjYAAAAALAeCxsAAAAA1mNhAwAAAMB6bB7gcxdffLGSrVixQsnWrVvnGt98883KnOeee07J9u7dq2SbNm2KokJ4JfwiZt1d2HUXM44aNSpuNXmtoaFByR599NFmn3f11Vcr2VNPPeVFSS3KwoULlezcc89Vsrffftuz98zLy1OyYcOGKdmFF17oGl966aWe1aDz/PPPK5nuwl/dheBoeX73u9+5xikpKTG/1pQpU063HPhQZmama7xmzRplzrXXXqtkX3/9tZLpNn4KPy7q/k7r2LGjkt14441Kpts8QDevNeKMDQAAAADrsbABAAAAYD0WNgAAAACsxzU2Fgr/HKiIyJgxY1zj22+/XZlz9OhRJduyZYuSbd682TUOv/EdzGnbtq2Sde3a1UAlzdNdTzNz5kwlmzVrlpLl5ua6xrqb0nbo0OE0qms9fvOb35guwYjS0tKI5l1//fVxrgSJpru58caNG2N6Ld0NkM8///yYXgt2yc/PV7L9+/fH9T11f5OVlZUpme4GoFwveBxnbAAAAABYj4UNAAAAAOuxsAEAAABgPRY2AAAAAKzH5gE+99577ynZqlWrlGzHjh2usW6jAJ3wm+aJiFxxxRURVodE013I6hfhF+zqNgVYvny5kulu5vjKK694VhdwKsOHDzddAjw2ePBgJfvf//7X7PN0F4svWbLEk5qASITfqFtEv1GALuMGncdxxgYAAACA9VjYAAAAALAeCxsAAAAA1mNhAwAAAMB6bB5gSEVFhZLNmzdPyXQXUVdXV8f0nmecof5z6+5an5zMetcEx3FOORYRWbNmjZI9++yz8SrppJ555hkle+KJJ1zjYDCozBk9erSSLV261LvCALR6Bw4cULKUlJRmn1dcXKxkHTp08KQmIBJFRUWmS7Aef8ECAAAAsB4LGwAAAADWi2phU1JSIv369ZP09HTp0qWLDB8+XPlI1eHDh6W4uFjOPvts6dChg4wcOVJqamo8LRqt11VXXUX/wZjZs2dzDIRRHANhEsdA+F1UC5uysjIpLi6WrVu3yqZNm+To0aMyePBgqa+vb5rzwAMPyLp162TlypVSVlYmVVVVMmLECM8LR+t0xx130H8w5q233uIYCKM4BsIkjoHwuyRHd4VyhPbv3y9dunSRsrIyueKKKyQYDErnzp1l2bJlcv3114uIyMcffywXXHCBlJeXy6WXXtrsa4ZCIQkEAhIMBiUjIyPW0ozSXdy/bNky13j+/PnKnE8//dSzGvr166dk06ZNUzI/38n+h3R9EY/+O9l7JcLKlStdY91dhHUbQNx1111KduuttyrZ2Wef7Rpv3bpVmfPiiy8q2bvvvqtklZWVSnbuuee6xrqf94QJE5Qs0n8Xk07WExwD/WvUqFFKtmLFCiXT3Vl+7NixcanpdCTqGGhb/91yyy1KtnjxYiXT3ak93L59+5Qs/LjWWnEMTIyNGzcq2ZAhQ5RM18/hf3t27tzZu8IMi6YnTusamxO7HnXs2FFERHbu3ClHjx6VwsLCpjk9e/aUvLw8KS8v175GQ0ODhEIh1wOIhBf9J0IPInYcA2ES/QfT6EH4TcwLm8bGRrn//vtl4MCBctFFF4nI8dViamqqZGZmuuZmZWWddIvikpISCQQCTY/c3NxYS0Ir4lX/idCDiA3HQJhE/8E0ehB+FPPCpri4WD744AN5+eWXT6uAqVOnSjAYbHroPuIChPOq/0ToQcSGYyBMov9gGj0IP4rpBp3jx4+X9evXy5YtW6Rbt25NeXZ2thw5ckRqa2tdq/WamhrJzs7WvlZaWpqkpaXFUkbC6Xb1+PDDD5Vs/PjxSvbxxx97Vkd+fr6STZ482TUeNmyYMqel3HjTy/4TsasHv/vuOyVbsGCBkq1atUrJAoGAa/zJJ5/EXMeAAQOUbNCgQa7x448/HvPr+11rPQa2RI2NjaZLiFpr7b89e/Yo2aZNm5RMd/1B+Pd4zz33KHOysrJiL66Vaa09GG//+c9/TJdgvaj+0nUcR8aPHy+rV6+W1157Tbp37+76ep8+faRNmzZSWlralFVUVMjnn38uBQUF3lSMVm3SpEn0H4zhGAjTOAbCJI6B8LuoztgUFxfLsmXLZO3atZKent70eclAICDt2rWTQCAgt912m0ycOFE6duwoGRkZcu+990pBQYEVOx/B/1asWEH/wZgHH3xQVq1aRQ/CGI6BMIljIPwuqoXNH//4RxE5foOwH1q0aJHcfPPNIiLyhz/8QZKTk2XkyJHS0NAgRUVFsnDhQk+KBYLBIP0HY1544QUR4RgIczgGwiSOgfC7qBY2kdzypm3btrJgwQLt5/6B09XcHub0H+Ipkj306UHEE8dAmMQxEH4X0+YBLdE333zjGutueqi7cNHLC70GDhyoZA8++KCSFRUVKVm7du08qwNmhH/+uH///sqc7du3R/Raum01dZtfhOvUqZOS6W4U+uyzz0ZUB+B3untrnPg/z/CX2tpaJYvkuCYikpOT4xrPnj3bi5IAT11++eVKFslJBXyvZWyTBQAAAKBVY2EDAAAAwHosbAAAAABYj4UNAAAAAOu1+M0Dtm3bpmSzZs1Ssh07drjGX3zxhad1nHnmma7xfffdp8yZNm2akrVv397TOuBfP7x7s4jIK6+8osz505/+pGRPPPFETO83YcIEJbv77ruVrEePHjG9PgAAiNzFF1+sZLrfwbqNq8Kzzp07e1eYRThjAwAAAMB6LGwAAAAAWI+FDQAAAADrsbABAAAAYL0Wv3nA6tWrI8oiceGFFyrZ0KFDlSwlJUXJJk2a5BpnZmbGVANaj65duyrZo48+GlEGtHZDhgxRshUrVhioBF7p2bOnkg0YMEDJ3njjjUSUAyTEQw89pGS33XZbs/Pmz5+vzNH9HdvScMYGAAAAgPVY2AAAAACwHgsbAAAAANZjYQMAAADAekmO4zimi/ihUCgkgUBAgsGgZGRkmC4HPpHIvqAHEY7+g2mJ6gv6DzocA80JhUJKdsMNNyjZpk2bXOORI0cqcxYtWqRk7du3P43qEiOanuCMDQAAAADrsbABAAAAYD0WNgAAAACs1+Jv0AkAAADYSHdNie5mw9OmTXONFy5cqMzR3dC7pd20kzM2AAAAAKzHwgYAAACA9VjYAAAAALAeCxsAAAAA1mPzAAAAAMASug0F5s2bd8pxa8EZGwAAAADWY2EDAAAAwHosbAAAAABYz3fX2DiOIyIioVDIcCXwkxP9cKI/4okeRDj6D6YlqgfpP+hwDIRJ0fSf7xY2dXV1IiKSm5truBL4UV1dnQQCgbi/hwg9CBX9B9Pi3YP0H06FYyBMiqT/kpxELL+j0NjYKFVVVZKeni51dXWSm5srlZWV2h0g/CwUCllbu4j/6nccR+rq6iQnJ0eSk+P7CcoTPeg4juTl5fnmZxAtv/0bRstP9dN/0fPTv18s/FZ/onqQ38H+4Lf6OQZGz2//htHyU/3R9J/vztgkJydLt27dREQkKSlJRI5va2f6hxorm2sX8Vf98f6/RCec6METpz799DOIBfV7g/6LDfV7JxE9yO9gf/FT/RwDY0P93oi0/9g8AAAAAID1WNgAAAAAsJ6vFzZpaWnyyCOPSFpamulSomZz7SL21+8F238G1G83279/6refzT8Dm2sXsb9+L9j+M6B+M3y3eQAAAAAARMvXZ2wAAAAAIBIsbAAAAABYj4UNAAAAAOuxsAEAAABgPd8ubBYsWCDnnXeetG3bVvLz82X79u2mS9LasmWLDB06VHJyciQpKUnWrFnj+rrjODJjxgzp2rWrtGvXTgoLC2Xv3r1mig1TUlIi/fr1k/T0dOnSpYsMHz5cKioqXHMOHz4sxcXFcvbZZ0uHDh1k5MiRUlNTY6jixKIH448ePDn6L/7ov5Oj/+KP/js1ejD+WmIP+nJhs3z5cpk4caI88sgjsmvXLunVq5cUFRXJV199Zbo0RX19vfTq1UsWLFig/fqsWbNk7ty58txzz8m2bdukffv2UlRUJIcPH05wpaqysjIpLi6WrVu3yqZNm+To0aMyePBgqa+vb5rzwAMPyLp162TlypVSVlYmVVVVMmLECINVJwY9mBj0oB79lxj0nx79lxj038nRg4nRInvQ8aH+/fs7xcXFTeNjx445OTk5TklJicGqmicizurVq5vGjY2NTnZ2tvP00083ZbW1tU5aWprz0ksvGajw1L766itHRJyysjLHcY7X2qZNG2flypVNc/71r385IuKUl5ebKjMh6EEz6MHj6D8z6L/j6D8z6L/v0YNmtIQe9N0ZmyNHjsjOnTulsLCwKUtOTpbCwkIpLy83WFn09u3bJ9XV1a7vJRAISH5+vi+/l2AwKCIiHTt2FBGRnTt3ytGjR1319+zZU/Ly8nxZv1foQXPoQfrPJPqP/jOJ/juOHjSnJfSg7xY2Bw4ckGPHjklWVpYrz8rKkurqakNVxeZEvTZ8L42NjXL//ffLwIED5aKLLhKR4/WnpqZKZmama64f6/cSPWgGPXgc/WcG/Xcc/WcG/fc9etCMltKDZ5guAP5QXFwsH3zwgbz55pumS0ErRQ/CJPoPJtF/MK2l9KDvzth06tRJUlJSlB0XampqJDs721BVsTlRr9+/l/Hjx8v69evl9ddfl27dujXl2dnZcuTIEamtrXXN91v9XqMHE48e/B79l3j03/fov8Sj/9zowcRrST3ou4VNamqq9OnTR0pLS5uyxsZGKS0tlYKCAoOVRa979+6SnZ3t+l5CoZBs27bNF9+L4zgyfvx4Wb16tbz22mvSvXt319f79Okjbdq0cdVfUVEhn3/+uS/qjxd6MHHoQRX9lzj0n4r+Sxz6T48eTJwW2YNGty44iZdfftlJS0tzFi9e7Hz00UfOnXfe6WRmZjrV1dWmS1PU1dU5u3fvdnbv3u2IiPPMM884u3fvdj777DPHcRznqaeecjIzM521a9c67733njNs2DCne/fuzqFDhwxX7jh33323EwgEnM2bNztffvll0+PgwYNNc371q185eXl5zmuvvea88847TkFBgVNQUGCw6sSgBxODHtSj/xKD/tOj/xKD/js5ejAxWmIP+nJh4ziOM2/ePCcvL89JTU11+vfv72zdutV0SVqvv/66IyLKY9y4cY7jHN/qb/r06U5WVpaTlpbmXH311U5FRYXZov+frm4RcRYtWtQ059ChQ84999zjnHXWWc6ZZ57pXHfddc6XX35prugEogfjjx48Ofov/ui/k6P/4o/+OzV6MP5aYg8mOY7jeHPuBwAAAADM8N01NgAAAAAQLRY2AAAAAKzHwgYAAACA9VjYAAAAALAeCxsAAAAA1mNhAwAAAMB6LGwAAAAAWI+FDQAAAADrsbABAAAAYD0WNgAAAACsx8IGAAAAgPVY2AAAAACw3v8BRW0HDjUJgGQAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "def reshape_2d_image(series):\n", " return series.values.reshape(28, 28)\n", "\n", "fig, axs = plt.subplots(nrows=1, ncols=5, figsize=(10, 5))\n", "for i in range(5):\n", " axs[i].imshow(reshape_2d_image(mnistIoData.inputs.iloc[i]), cmap=\"binary\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Applying Predefined Models\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We create an evaluator in order to test the performance of our models, randomly splitting the data." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:33.905592Z", "iopub.status.busy": "2024-08-13T22:11:33.905022Z", "iopub.status.idle": "2024-08-13T22:11:33.926342Z", "shell.execute_reply": "2024-08-13T22:11:33.925657Z" } }, "outputs": [], "source": [ "evaluator_params = sensai.evaluation.ClassificationEvaluatorParams(fractional_split_test_fraction=0.2)\n", "eval_util = sensai.evaluation.ClassificationModelEvaluation(mnistIoData, evaluator_params=evaluator_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "One pre-defined model we could try is a simple multi-layer perceptron. A PyTorch-based implementation is provided via class `MultiLayerPerceptronVectorClassificationModel`. This implementation supports CUDA-accelerated computations (on Nvidia GPUs), yet we shall stick to CPU-based computation (cuda=False) in this tutorial." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:33.929919Z", "iopub.status.busy": "2024-08-13T22:11:33.929384Z", "iopub.status.idle": "2024-08-13T22:11:35.487598Z", "shell.execute_reply": "2024-08-13T22:11:35.486912Z" } }, "outputs": [], "source": [ "import sensai.torch\n", "\n", "nn_optimiser_params = sensai.torch.NNOptimiserParams(early_stopping_epochs=2, batch_size=54)\n", "torch_mlp_model = sensai.torch.models.MultiLayerPerceptronVectorClassificationModel(hidden_dims=(50, 20),\n", " cuda=False, normalisation_mode=sensai.torch.NormalisationMode.MAX_ALL,\n", " nn_optimiser_params=nn_optimiser_params, p_dropout=0.0) \\\n", " .with_name(\"MLP\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Neural networks work best on **normalised inputs**, so we have opted to apply basic normalisation by specifying a normalisation mode which will transforms inputs by dividing by the maximum value found across all columns in the training data. For more elaborate normalisation options, we could have used a data frame transformer (DFT), particularly `DFTNormalisation` or `DFTSkLearnTransformer`.\n", "\n", "sensAI's default **neural network training algorithm** is based on early stopping, which involves checking, in regular intervals, the performance of the model on a validation set (which is split from the training set) and ultimately selecting the model that performed best on the validation set. You have full control over the loss evaluation method used to select the best model (by passing a respective `NNLossEvaluator` instance to NNOptimiserParams) as well as the method that is used to split the training set into the actual training set and the validation set (by adding a `DataFrameSplitter` to the model or using a custom `TorchDataSetProvider`).\n", "\n", "Given the vectorised nature of our MNIST dataset, we can apply any type of model which can accept the numeric inputs. Let's compare the neural network we defined above against another pre-defined model, which is based on a scikit-learn implementation and uses decision trees rather than neural networks." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:35.491175Z", "iopub.status.busy": "2024-08-13T22:11:35.490756Z", "iopub.status.idle": "2024-08-13T22:11:35.514053Z", "shell.execute_reply": "2024-08-13T22:11:35.513380Z" } }, "outputs": [], "source": [ "random_forest_model = sensai.sklearn.classification.SkLearnRandomForestVectorClassificationModel(\n", " min_samples_leaf=1,\n", " n_estimators=10) \\\n", " .with_name(\"RandomForest\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's compare the two models using our evaluation utility." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:35.517249Z", "iopub.status.busy": "2024-08-13T22:11:35.516835Z", "iopub.status.idle": "2024-08-13T22:11:56.404274Z", "shell.execute_reply": "2024-08-13T22:11:56.403515Z" } }, "outputs": [], "source": [ "eval_util.compare_models([random_forest_model, torch_mlp_model]);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Both models perform reasonably well." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating a Custom CNN Model\n", "\n", "Given that this is an image recognition problem, it can be sensible to apply convolutional neural networks (CNNs), which can analyse patches of the image in order to generate more high-level features from them.\n", "Specifically, we shall apply a neural network model which uses multiple convolutions, a max-pooling layer and a multi-layer perceptron at the end in order to produce the classification.\n", "\n", "For classification and regression, sensAI provides the fundamental classes `TorchVectorClassificationModel` and `TorchVectorRegressionModel` respectively. Ultimately, these classes will wrap an instance of `torch.nn.Module`, the base class for neural networks in PyTorch.\n", "\n", "#### Wrapping a Custom torch.nn.Module Instance\n", "\n", "If we already had an implementation of a `torch.nn.Module`, it can be straightforwardly adapted to become a sensAI ``VectorModel``.\n", "\n", "Let's say we had the following implementation of a torch module, which performs the steps described above.\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:56.409533Z", "iopub.status.busy": "2024-08-13T22:11:56.409185Z", "iopub.status.idle": "2024-08-13T22:11:56.452647Z", "shell.execute_reply": "2024-08-13T22:11:56.451947Z" } }, "outputs": [], "source": [ "import torch\n", "\n", "class MnistCnnModule(torch.nn.Module):\n", " def __init__(self, image_dim: int, output_dim: int, num_conv: int, kernel_size: int, pooling_kernel_size: int,\n", " mlp_hidden_dims: Sequence[int], output_activation_fn: sensai.torch.ActivationFunction, p_dropout=0.0):\n", " super().__init__()\n", " k = kernel_size\n", " p = pooling_kernel_size\n", " self.cnn = torch.nn.Conv2d(1, num_conv, (k, k))\n", " self.pool = torch.nn.MaxPool2d((p, p))\n", " self.dropout = torch.nn.Dropout(p=p_dropout)\n", " reduced_dim = (image_dim - k + 1) / p\n", " if int(reduced_dim) != reduced_dim:\n", " raise ValueError(f\"Pooling kernel size {p} is not a divisor of post-convolution dimension {image_dim - k + 1}\")\n", " self.mlp = sensai.torch.models.MultiLayerPerceptron(num_conv * int(reduced_dim) ** 2, output_dim, mlp_hidden_dims,\n", " output_activation_fn=output_activation_fn.get_torch_function(),\n", " hid_activation_fn=sensai.torch.ActivationFunction.RELU.get_torch_function(),\n", " p_dropout=p_dropout)\n", "\n", " def forward(self, x):\n", " x = self.cnn(x.unsqueeze(1))\n", " x = self.pool(x)\n", " x = x.view(x.shape[0], -1)\n", " x = self.dropout(x)\n", " return self.mlp(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Since this module requires 2D images as input, we will need a component that transforms the vector input that is given in our data frame into a tensor that will serve as input to the module.\n", "In sensAI, the abstraction for this purpose is a ``sensai.torch.Tensoriser``. A **Tensoriser** can, in principle, perform arbitrary computations in order to produce, from a data frame with N rows, one or more tensors of length N (first dimension equal to N) that will ultimately be fed to the neural network.\n", "\n", "Luckily, for the case at hand, we already have the function ``reshape_2d_image`` from above to assist in the implementation of the tensoriser." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:56.456446Z", "iopub.status.busy": "2024-08-13T22:11:56.455805Z", "iopub.status.idle": "2024-08-13T22:11:56.486603Z", "shell.execute_reply": "2024-08-13T22:11:56.486109Z" } }, "outputs": [], "source": [ "class ImageReshapingInputTensoriser(sensai.torch.RuleBasedTensoriser):\n", " def _tensorise(self, df: pd.DataFrame) -> Union[torch.Tensor, List[torch.Tensor]]:\n", " images = [reshape_2d_image(row) for _, row in df.iterrows()]\n", " return torch.tensor(np.stack(images)).float() / 255" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "In this case, we derived the class from ``RuleBasedTensoriser`` rather than ``Tensoriser``, because our tensoriser does not require fitting. We additionally took care of the normalisation.\n", "\n", "Now we have all we need to create a sensAI ``TorchVectorClassificationModel`` that will work on the input/output data we loaded earlier." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:56.489560Z", "iopub.status.busy": "2024-08-13T22:11:56.489086Z", "iopub.status.idle": "2024-08-13T22:11:56.519771Z", "shell.execute_reply": "2024-08-13T22:11:56.519234Z" } }, "outputs": [], "source": [ "cnn_module = MnistCnnModule(28, 10, 32, 5, 2, (200, 20), sensai.torch.ActivationFunction.LOG_SOFTMAX)\n", "nn_optimiser_params = sensai.torch.NNOptimiserParams(\n", " optimiser=sensai.torch.Optimiser.ADAMW,\n", " optimiser_lr=0.01,\n", " batch_size=1024,\n", " early_stopping_epochs=3)\n", "cnn_model_from_module = sensai.torch.TorchVectorClassificationModel.from_module(\n", " cnn_module, sensai.torch.ClassificationOutputMode.LOG_PROBABILITIES,\n", " cuda=False, nn_optimiser_params=nn_optimiser_params) \\\n", " .with_input_tensoriser(ImageReshapingInputTensoriser()) \\\n", " .with_name(\"CNN\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We have now fully defined all the necessary parameters, including parameters controlling the training of the model.\n", "\n", "We are now ready to evaluate the model." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:11:56.522778Z", "iopub.status.busy": "2024-08-13T22:11:56.522411Z", "iopub.status.idle": "2024-08-13T22:12:57.336531Z", "shell.execute_reply": "2024-08-13T22:12:57.335675Z" } }, "outputs": [], "source": [ "eval_util.perform_simple_evaluation(cnn_model_from_module);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Creating an Input-/Output-Adaptive Custom Model\n", "\n", "While the above approach allows us to straightforwardly encapsulate a ``torch.nn.Module``, it really doesn't follow sensAI's principle of adapting model hyperparameters based on the inputs and outputs we receive during training - whenever possible. Notice that in the above example, we had to hard-code the image dimension (``28``) as well as the number of classes (``10``), even though these parameters could have been easily determined from the data. Especially in other domains where feature engineering is possible, we might want to experiment with different combinations of features, and therefore automatically adapting to inputs is key if we want to avoid editing the model hyperparameters time and time again; similarly, we might change the set of target labels in our classification problem and the model should simply adapt to a changed output dimension.\n", "\n", "To design a model that can fully adapt to the inputs and outputs, we can simply subclass ``TorchVectorClassificationModel``, where the late instantiation of the underlying model is catered for. Naturally, delayed construction of the underlying model necessitates the use of factories and thus results in some indirections. \n", "\n", "If we had designed the above model to be within the sensAI ``VectorModel`` realm from the beginning, here's what we might have written:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:12:57.340783Z", "iopub.status.busy": "2024-08-13T22:12:57.340509Z", "iopub.status.idle": "2024-08-13T22:12:57.390312Z", "shell.execute_reply": "2024-08-13T22:12:57.389526Z" } }, "outputs": [], "source": [ "import torch\n", "\n", "class CnnModel(sensai.torch.TorchVectorClassificationModel):\n", " def __init__(self, cuda: bool, kernel_size: int, num_conv: int, pooling_kernel_size: int, mlp_hidden_dims: Sequence[int],\n", " nn_optimiser_params: sensai.torch.NNOptimiserParams, p_dropout=0.0):\n", " self.cuda = cuda\n", " self.output_activation_fn = sensai.torch.ActivationFunction.LOG_SOFTMAX\n", " self.kernel_size = kernel_size\n", " self.num_conv = num_conv\n", " self.pooling_kernel_size = pooling_kernel_size\n", " self.mlp_hidden_dims = mlp_hidden_dims\n", " self.p_dropout = p_dropout\n", " super().__init__(sensai.torch.ClassificationOutputMode.for_activation_fn(self.output_activation_fn),\n", " torch_model_factory=functools.partial(self.VectorTorchModel, self),\n", " nn_optimiser_params=nn_optimiser_params)\n", "\n", " class VectorTorchModel(sensai.torch.VectorTorchModel):\n", " def __init__(self, parent: \"CnnModel\"):\n", " super().__init__(parent.cuda)\n", " self._parent = parent\n", "\n", " def create_torch_module_for_dims(self, input_dim: int, output_dim: int) -> torch.nn.Module:\n", " return self.Module(int(np.sqrt(input_dim)), output_dim, self._parent)\n", "\n", " class Module(torch.nn.Module):\n", " def __init__(self, image_dim, output_dim, parent: \"CnnModel\"):\n", " super().__init__()\n", " k = parent.kernel_size\n", " p = parent.pooling_kernel_size\n", " self.cnn = torch.nn.Conv2d(1, parent.num_conv, (k, k))\n", " self.pool = torch.nn.MaxPool2d((p, p))\n", " self.dropout = torch.nn.Dropout(p=parent.p_dropout)\n", " reduced_dim = (image_dim - k + 1) / p\n", " if int(reduced_dim) != reduced_dim:\n", " raise ValueError(f\"Pooling kernel size {p} is not a divisor of post-convolution dimension {image_dim - k + 1}\")\n", " self.mlp = sensai.torch.models.MultiLayerPerceptron(parent.num_conv * int(reduced_dim) ** 2, output_dim, parent.mlp_hidden_dims,\n", " output_activation_fn=parent.output_activation_fn.get_torch_function(),\n", " hid_activation_fn=sensai.torch.ActivationFunction.RELU.get_torch_function(),\n", " p_dropout=parent.p_dropout)\n", "\n", " def forward(self, x):\n", " x = self.cnn(x.unsqueeze(1))\n", " x = self.pool(x)\n", " x = x.view(x.shape[0], -1)\n", " x = self.dropout(x)\n", " return self.mlp(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "It is only insignificantly more code than in the previous implementation.\n", "The outer class, which provides the sensAI `VectorModel` features, serves mainly to hold the parameters, and the inner class inheriting from `VectorTorchModel` serves as a factory for the `torch.nn.Module`, providing us with the input and output dimensions (number of input columns and number of classes respectively) based on the data, thus enabling the model to adapt. If we had required even more adaptiveness, we could have learnt more about the data from within the fitting process of a custom input tensoriser (i.e. we could have added an inner ``Tensoriser`` class, which could have derived further hyperparameters from the data in its implementation of the fitting method.)\n", "\n", "Let's instantiate our model and evaluate it." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:12:57.395057Z", "iopub.status.busy": "2024-08-13T22:12:57.394780Z", "iopub.status.idle": "2024-08-13T22:13:56.820491Z", "shell.execute_reply": "2024-08-13T22:13:56.819667Z" } }, "outputs": [], "source": [ "cnn_model = CnnModel(cuda=False, kernel_size=5, num_conv=32, pooling_kernel_size=2, mlp_hidden_dims=(200,20),\n", " nn_optimiser_params=nn_optimiser_params) \\\n", " .with_name(\"CNN'\") \\\n", " .with_input_tensoriser(ImageReshapingInputTensoriser())\n", "\n", "eval_data = eval_util.perform_simple_evaluation(cnn_model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Our CNN models do improve upon the MLP model we evaluated earlier. Let's do a comparison of all the models we trained thus far:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:13:56.825038Z", "iopub.status.busy": "2024-08-13T22:13:56.824522Z", "iopub.status.idle": "2024-08-13T22:14:01.322218Z", "shell.execute_reply": "2024-08-13T22:14:01.321542Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
accuracybalancedAccuracy
model_name
MLP0.9622500.961897
CNN0.9783330.978435
CNN'0.9771670.977261
RandomForest0.9466670.945917
\n", "
" ], "text/plain": [ " accuracy balancedAccuracy\n", "model_name \n", "MLP 0.962250 0.961897\n", "CNN 0.978333 0.978435\n", "CNN' 0.977167 0.977261\n", "RandomForest 0.946667 0.945917" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "comparison_data = eval_util.compare_models([torch_mlp_model, cnn_model_from_module, cnn_model, random_forest_model], fit_models=False)\n", "comparison_data.results_df" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that any differences between the two CNN models are due only to randomness in the parameter initialisation; they are functionally identical.\n", "\n", "Could the CNN model have produced even better results? Let's take a look at some examples where the CNN model went wrong by inspecting the evaluation data that was returned earlier." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "execution": { "iopub.execute_input": "2024-08-13T22:14:01.326837Z", "iopub.status.busy": "2024-08-13T22:14:01.326398Z", "iopub.status.idle": "2024-08-13T22:14:02.654712Z", "shell.execute_reply": "2024-08-13T22:14:02.653930Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2YAAAN6CAYAAADlwVoeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuNSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/xnp5ZAAAACXBIWXMAAA9hAAAPYQGoP6dpAACXvElEQVR4nOzdeXQUZfr28avZmrAkMRASQgIEZVFAVJRdAsqPEFEBWcSVxQ0FFXDFGWVxQWFGGQXBbUARRAHBkVGUHZeAgDoMKMiqQUgQMAlrCOR5/+ClhyahKp3upjrJ93NOn0Pqrq56upK+wp2qrsdljDECAAAAADimjNMDAAAAAIDSjsYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMasmJg2bZpcLpd27twZ8G3XrVtX/fv3D/h2C6t///6qW7eu17JDhw7p7rvvVmxsrFwul4YOHaqdO3fK5XJp2rRpAdt3MI8rUBosX75cLpdLy5cvD/i2O3TooA4dOgR8u4U1atQouVwur2UnTpzQ448/roSEBJUpU0bdu3eXJLlcLo0aNSpg+w7mcQVKA7KpuySyqbihMfPBli1b1LdvX8XHx6tSpUpq1KiRxowZoyNHjjg9tBLnhRde0LRp03T//fdr+vTpuuOOO5weUlCtWbNGQ4YMUePGjVW5cmXVrl1bffr00S+//OL00BDi+Nk5v/75z39q/Pjx6tWrl959910NGzbM6SEF3datW9WrVy9dcMEFqlSpktq1a6dly5Y5PSyEOLLp/CqN2XSmGTNmyOVyqUqVKk4PxS8uY4xxehDFQVpami699FJFRERo0KBBioqKUmpqqqZNm6Ybb7xRn3zySVD3f/LkSeXm5srtduf7K4m/6tatqw4dOgT0TJQvcnNzlZeXJ7fb7VnWqlUrlStXTl9//bVnmTFGOTk5Kl++vMqWLRuQfU+bNk0DBgzQjh078p21O5969eqlb775Rr1799all16q9PR0TZw4UYcOHdKqVavUpEkTx8aG0Ob0z05eXp6OHz+uChUqqEyZwP6t7/RfpJ36y+yJEyd04sQJVaxY0bOsb9+++vrrr7Vr1y6vdY8dO6Zy5cqpXLlyAdn38uXL1bFjRy1btszRv8ynpaXpiiuuUNmyZfXQQw+pcuXKmjp1qjZu3KglS5aoffv2jo0NoY1sCh6yyduhQ4fUsGFDZWVleb4urgLzXSoFpk+frszMTH399ddq3LixJOnee+9VXl6e3nvvPf3555+64IILgrb/smXLBqwZCTXly5fPt2zv3r265JJLvJa5XC6vECpJhg8frpkzZ6pChQqeZTfffLOaNm2qF198Ue+//76Do0Moc/pnp0yZMiX2fVnQf2b27t2ryMjIfOuW1GPw4osvKjMzUxs2bFDDhg0lSffcc48aNWqkYcOGad26dQ6PEKGKbAoessnbc889p6pVq6pjx46aP3++08PxC5cyFlJ2drYkKSYmxmt5zZo1VaZMGa/gKUiHDh3UpEkTrV+/XklJSapUqZIuuugizZkzR5K0YsUKtWzZUmFhYWrYsKEWL17s9fyCPgu1du1aJScnq3r16goLC1NiYqIGDhzo9by8vDz94x//UNOmTVWxYkVFR0erS5cuWrt27TnHeuDAAT366KNq2rSpqlSpovDwcKWkpOg///lPvnVfe+01NW7cWJUqVdIFF1ygK6+8UjNnzvTUDx48qKFDh6pu3bpyu92qUaOG/u///k/ff/+9Z50zP2N2+trlHTt26N///rdcLpfndZ/rM2abNm1Sr169FBUVpYoVK+rKK6/Uv/71r3xj3bhxo6655hqFhYUpPj5ezz33nPLy8s55HM60fv169e/fX/Xq1VPFihUVGxurgQMHav/+/V7rFeb1FqRNmzb5fobq16+vxo0b6+effy7UGFE6+fuz079/f1WpUkW//fabrr/+elWpUkW1atXSpEmTJEn//e9/dc0116hy5cqqU6eO1/tbKvjzBlu2bFHPnj0VGxurihUrKj4+Xn379vX8NfO0999/Xy1atPDkR/v27fXll1+ec6zHjx/XM888o+bNmysiIkKVK1fW1VdfXeBldbNmzVLz5s1VtWpVhYeHq2nTpvrHP/7hqefm5mr06NGqX7++KlasqGrVqqldu3ZatGiRZ50zP8dxOn+WLVumjRs3erLp9Osu6HMcv//+uwYOHKiYmBi53W41btxY//znP/ONddeuXerevbsqV66sGjVqaNiwYcrJyTnncTjTr7/+qgceeEANGzZUWFiYqlWrpt69e+f73GxhXm9BvvrqK11++eWepkySKlWqpBtvvFHff/+9tmzZUqhxovQhm8imYGbTaVu2bNErr7yil19+OWBnBZ1U/F/BedKhQwe99NJLuuuuuzR69GhVq1ZN3377rSZPnuy5vMPOn3/+qeuvv159+/ZV7969NXnyZPXt21czZszQ0KFDNWjQIN16662ea4TT0tJUtWrVAre1d+9ede7cWdHR0XryyScVGRmpnTt36uOPP/Za76677tK0adOUkpKiu+++WydOnNBXX32lVatW6corryxw29u3b9f8+fPVu3dvJSYmKiMjQ2+88YaSkpL0008/KS4uTpL01ltv6aGHHlKvXr308MMP69ixY1q/fr1Wr16tW2+9VZI0aNAgzZkzR0OGDNEll1yi/fv36+uvv9bPP/+sK664It++L774Yk2fPl3Dhg1TfHy8HnnkEUlSdHS0/vjjj3zrb9y4UW3btlWtWrX05JNPqnLlyvroo4/UvXt3zZ07Vz169JAkpaenq2PHjjpx4oRnvTfffFNhYWG23zdJWrRokbZv364BAwYoNjZWGzdu1JtvvqmNGzdq1apVnoD09fVaMcYoIyPDc4YWKCxff3ZOnjyplJQUtW/fXuPGjdOMGTM0ZMgQVa5cWX/5y19022236aabbtKUKVN05513qnXr1kpMTCxwW8ePH1dycrJycnL04IMPKjY2Vr///rsWLFigzMxMRURESJJGjx6tUaNGqU2bNhozZowqVKig1atXa+nSpercuXOB287Oztbbb7+tW265Rffcc48OHjyod955R8nJyfruu+902WWXSTr1fr3lllt07bXX6qWXXpIk/fzzz/rmm2/08MMPSzr1H5uxY8fq7rvvVosWLZSdna21a9fq+++/1//93//l23d0dLSmT5+u559/XocOHdLYsWMlncqsgmRkZKhVq1ZyuVwaMmSIoqOj9fnnn+uuu+5Sdna2hg4dKkk6evSorr32Wv3222966KGHFBcXp+nTp2vp0qWF+t6tWbNG3377refzzzt37tTkyZPVoUMH/fTTT6pUqVKRXu9pOTk5BV4Ncnq769atU/369Qs1VoBsIpsClU2nDR06VB07dtR1112njz76qFBjC2kGhfbss8+asLAwI8nz+Mtf/lKo5yYlJRlJZubMmZ5lmzZtMpJMmTJlzKpVqzzLv/jiCyPJTJ061bNs6tSpRpLZsWOHMcaYefPmGUlmzZo159zn0qVLjSTz0EMP5avl5eV5/l2nTh3Tr18/z9fHjh0zJ0+e9Fp/x44dxu12mzFjxniWdevWzTRu3NjydUdERJjBgwdbrtOvXz9Tp04dr2V16tQxXbt2zTeGs4/Ltddea5o2bWqOHTvm9dratGlj6tev71k2dOhQI8msXr3as2zv3r0mIiLC67iey5EjR/It++CDD4wks3LlSs+ywrzewpo+fbqRZN55552AbA+lhy8/O/369TOSzAsvvOBZ9ueff5qwsDDjcrnMrFmzPMtPZ9bIkSM9y5YtW2YkmWXLlhljjPnhhx+MJDN79uxz7nPLli2mTJkypkePHvmy5sxsSkpKMklJSZ6vT5w4YXJycrzW//PPP01MTIwZOHCgZ9nDDz9swsPDzYkTJ845hmbNmuXLmLONHDnSnP1rMikpqcDcO/u43HXXXaZmzZpm3759Xuv17dvXREREeDJlwoQJRpL56KOPPOscPnzYXHTRRV7H9VwKyqbU1FQjybz33nueZYV5vQW54YYbTGRkpMnOzvZa3rp1ayPJ/O1vf/N5myi9yCayKVDZZIwxCxYsMOXKlTMbN240xpz6malcuXKRthUquJTRB3Xr1lX79u315ptvau7cuRo4cKBeeOEFTZw4sVDPr1Klivr27ev5umHDhoqMjNTFF1+sli1bepaf/vf27dvPua3T1xEvWLBAubm5Ba4zd+5cuVwujRw5Ml/N6gYibrfb80HZkydPav/+/apSpYoaNmzodUleZGSkdu3apTVr1liOc/Xq1dq9e/c51ymqAwcOaOnSperTp48OHjyoffv2ad++fdq/f7+Sk5O1ZcsW/f7775Kkzz77TK1atVKLFi08z4+OjtZtt91WqH2deWbt2LFj2rdvn1q1aiVJ+Y5JIF7vpk2bNHjwYLVu3Vr9+vXza1soXYr6s3P33Xd7/h0ZGamGDRuqcuXK6tOnj2f56cyyyqbTf3X+4osvznnH2vnz5ysvL0/PPPNMvg/lW2VT2bJlPZdG5eXl6cCBAzpx4oSuvPLKfO/Dw4cPW14KExkZqY0bNwblUjxjjObOnasbbrhBxhhPNu3bt0/JycnKysryjPezzz5TzZo11atXL8/zK1WqpHvvvbdQ+zozm3Jzc7V//35ddNFFioyMzHdMivJ677//fmVmZurmm2/WDz/8oF9++UVDhw71XA5/9OhRn7aH0otsIpsCmU3Hjx/XsGHDNGjQoHz3JCjOaMwKadasWbr33nv19ttv65577tFNN92kd955R/369dMTTzyR77NGBYmPj8/3xo6IiFBCQkK+ZdKpSx/PJSkpST179tTo0aNVvXp1devWTVOnTvW69nfbtm2Ki4tTVFSULy9VeXl5euWVV1S/fn253W5Vr15d0dHRWr9+vdd12E888YSqVKmiFi1aqH79+ho8eLC++eYbr22NGzdOGzZsUEJCglq0aKFRo0ZZBqcvtm7dKmOMnn76aUVHR3s9Tjeje/fulXTqWueCLrc583MTVg4cOKCHH35YMTExCgsLU3R0tOdyiTOPSSBeb3p6urp27aqIiAjNmTOnxN70BYFX1J+d058/PVNERMQ5M8sqmxITEzV8+HC9/fbbql69upKTkzVp0iSv98m2bdtUpkyZIv0yfffdd3XppZd6PosQHR2tf//7317bf+CBB9SgQQOlpKQoPj5eAwcO1MKFC722M2bMGGVmZqpBgwZq2rSpHnvsMa1fv97n8RTkjz/+UGZmpt5888182TRgwABJ3tl00UUX5TvOhc2mo0eP6plnnlFCQoJXXmdmZnodk6K+3pSUFL322mtauXKlrrjiCjVs2FD//ve/9fzzz0tSsb81Nc4PsukUsilw2fTKK69o3759Gj16dGFffrFAY1ZIr7/+ui6//HLFx8d7Lb/xxht15MgR/fDDD7bbOFcQnWu5sZjJwOVyac6cOUpNTdWQIUM8H+Rs3ry537cJfeGFFzR8+HC1b99e77//vr744gstWrRIjRs39rpZxsUXX6zNmzdr1qxZateunebOnat27dp5naHr06ePtm/frtdee01xcXEaP368GjdurM8//9yvMUryjOXRRx/VokWLCnxcdNFFfu9HOvU63nrrLQ0aNEgff/yxvvzyS0+YnnlM/H29WVlZSklJUWZmphYuXOj5PB9gx5+fnUBmkyT9/e9/1/r16/XUU0/p6NGjeuihh9S4ceN8t3H21fvvv6/+/fvrwgsv1DvvvKOFCxdq0aJFuuaaa7zehzVq1NCPP/6of/3rX7rxxhu1bNkypaSkeP2Vvn379tq2bZv++c9/qkmTJnr77bd1xRVX6O233/ZrjNL/MuH2228/Zza1bdvW7/1I0oMPPqjnn39effr00UcffaQvv/xSixYtUrVq1byOiT+vd8iQIcrIyNC3336rtWvXatOmTZ4/IDZo0CAgrwMlF9lENgU6m7KysvTcc8/pnnvuUXZ2tucGcYcOHZIxRjt37vQ0mMWOYxdRFjMNGjQwLVu2zLf8ww8/NJLM559/bvn8c137W9BnqYw5dU3wmZ9VOvszZgWZMWOGkWTeeustY4wxgwcPNi6Xy+zfv99ybGd/xqxZs2amY8eO+darVauW1zXVZ8vJyTFdu3Y1ZcuWNUePHi1wnYyMDFOrVi3Ttm1bz7KifsYsIyPDSDIjRoywfH3GnPr+tWrVKt/yBx54wPa4HjhwwEgyo0eP9lr+yy+/5Lt2+2wFvd5zOXr0qLn66qtNpUqVzLfffmu7PnCaPz8757omv7CZdfbnOAryzTffeH0md/z48UaS+eGHHyzHdvbnOLp162bq1avn9VkPY4xp06ZNvgw508mTJ819991nJJktW7YUuM7BgwfN5ZdfbmrVquVZVtTPcZw4ccJUrVrV3HLLLZavzxhjOnfubOLi4vK9pnHjxhXqcxwRERFmwIABXsuOHj1qypYt65XrZyvo9fqid+/eJiwszGRmZhbp+SgdyCay6UyByqbT/x+0enTr1s3uJYYkzpgVUoMGDTzX15/pgw8+UJkyZXTppZee1/H8+eef+f4ydPquP6cvZ+zZs6eMMQWe5j37uWcqW7Zsvvrs2bM9n9c67ezLNytUqKBLLrlExhjl5ubq5MmT+W5BW6NGDcXFxRX6dqtWatSooQ4dOuiNN97Qnj178tXPvIvjddddp1WrVum7777zqs+YMcN2P6f/Mnf2MZkwYYLX1/683pMnT+rmm29WamqqZs+erdatW9uOC5BC72cnOztbJ06c8FrWtGlTlSlTxvM+6N69u8qUKaMxY8bkm7LCLpvOXmf16tVKTU31Wu/sbDozo0+P4ex1qlSpoosuuigg2VS2bFn17NlTc+fO1YYNG/LVz86m3bt3e6ZOkaQjR47ozTffLPS+zj5mr732mk6ePOm1LJCv99tvv9XHH3+su+66y3PmDDgb2UQ2BSubatSooXnz5uV7dOzYURUrVtS8efM0YsSIQo0z1HC7/EJ67LHH9Pnnn+vqq6/WkCFDVK1aNS1YsECff/657r777vN+ydm7776r119/XT169NCFF16ogwcP6q233lJ4eLiuu+46SVLHjh11xx136NVXX9WWLVvUpUsX5eXl6auvvlLHjh01ZMiQArd9/fXXa8yYMRowYIDatGmj//73v5oxY4bq1avntV7nzp0VGxurtm3bKiYmRj///LMmTpyorl27qmrVqsrMzFR8fLx69eqlZs2aqUqVKlq8eLHWrFmjv//97wE5DpMmTVK7du3UtGlT3XPPPapXr54yMjKUmpqqXbt2eeZee/zxxzV9+nR16dJFDz/8sOd2+XXq1LG9ljk8PNxzq97c3FzVqlVLX375pXbs2OG13sGDB4v8eh955BH961//0g033KADBw7km3jz9ttvL8LRQWkQaj87S5cu1ZAhQ9S7d281aNBAJ06c0PTp0z3/IZCkiy66SH/5y1/07LPP6uqrr9ZNN90kt9utNWvWKC4uznO757Ndf/31+vjjj9WjRw917dpVO3bs0JQpU3TJJZd4XcJ9991368CBA7rmmmsUHx+vX3/9Va+99pouu+wyzy2kL7nkEnXo0EHNmzdXVFSU1q5d65nqIhBefPFFLVu2TC1bttQ999yjSy65RAcOHND333+vxYsX68CBA5JOTdY8ceJE3XnnnVq3bp1q1qyp6dOne24lbef666/X9OnTFRERoUsuuUSpqalavHixqlWr5rVeUV/vr7/+qj59+ujGG2/0TBUyZcoUXXrppXrhhReKdnBQKpBNZFOwsqlSpUrq3r17vuXz58/Xd999V2Ct2HDmRF3xtHr1apOSkmJiY2NN+fLlTYMGDczzzz9vcnNzbZ8b6EsZv//+e3PLLbeY2rVrG7fbbWrUqGGuv/56s3btWq/tnDhxwowfP940atTIVKhQwURHR5uUlBSzbt06rzGcfbv8Rx55xNSsWdOEhYWZtm3bmtTU1Hyn7t944w3Tvn17U61aNeN2u82FF15oHnvsMZOVlWWMOXVp42OPPWaaNWtmqlataipXrmyaNWtmXn/9da8x+nO7fGOM2bZtm7nzzjs935datWqZ66+/3syZM8drvfXr15ukpCRTsWJFU6tWLfPss8+ad955p1C3y9+1a5fp0aOHiYyMNBEREaZ3795m9+7dXpcIFPb1FuT0dArnegDn4u/PTqAvF9q+fbsZOHCgufDCC03FihVNVFSU6dixo1m8eHG+bf3zn/80l19+uXG73eaCCy4wSUlJZtGiRV5jODNz8vLyzAsvvGDq1Klj3G63ufzyy82CBQvyZcicOXNM586dTY0aNUyFChVM7dq1zX333Wf27NnjWee5554zLVq0MJGRkSYsLMw0atTIPP/88+b48eOedfy5JbUxpy5lHjx4sElISDDly5c3sbGx5tprrzVvvvmm13q//vqrufHGG02lSpVM9erVzcMPP2wWLlxYqMuF/vzzTzNgwABTvXp1U6VKFZOcnGw2bdqUL9cL83oLcuDAAdOtWzcTGxtrKlSoYBITE80TTzyR7/b5wNnIJrIpmNlUkJJwu3yXMTaflAQAAAAABBWfMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOCzkJpjOy8vT7t27VbVqVblcLqeHA6AAxhgdPHhQcXFxKlOmdPx9h2wCQh/ZRDYBoajQ2RSsCdImTpzomWivRYsWZvXq1YV6XlpamuWEhDx48AidR1paWrAiJGjIJh48Sv6juGVTUXPJGLKJB4/i9LDLpqCcMfvwww81fPhwTZkyRS1bttSECROUnJyszZs3q0aNGpbPrVq1qiQpLS1N4eHhwRgeAD9lZ2crISHB834tLsgmoGQrjtnkTy5JZBNQHBQ2m1zGGBPonbds2VJXXXWVJk6cKOnUafaEhAQ9+OCDevLJJy2fm52drYiICGVlZREwQIgqru9Tsgko2Yrj+9SfXJKK52sGSpvCvk8DfgH28ePHtW7dOnXq1Ol/OylTRp06dVJqamqgdwcAhUI2AQg15BKAMwX8UsZ9+/bp5MmTiomJ8VoeExOjTZs25Vs/JydHOTk5nq+zs7MDPSQAIJsAhBxfc0kim4CSzPFbFo0dO1YRERGeR0JCgtNDAgCyCUBIIpuAkivgjVn16tVVtmxZZWRkeC3PyMhQbGxsvvVHjBihrKwszyMtLS3QQwIAsglAyPE1lySyCSjJAt6YVahQQc2bN9eSJUs8y/Ly8rRkyRK1bt063/put1vh4eFeDwAINLIJQKjxNZcksgkoyYJyu/zhw4erX79+uvLKK9WiRQtNmDBBhw8f1oABA4KxOwAoFLIJQKghlwCcFpTG7Oabb9Yff/yhZ555Runp6brsssu0cOHCfB9uBYDziWwCEGrIJQCnBWUeM38wHwcQ+krj+7Q0vmaguCmN79PS+JqB4saxecwAAAAAAL6hMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHBYOacHgNC3evVq23Wuu+46y3qTJk0s63/9619t99GxY0fLerly/DgDAACgeOKMGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwJn6CMjMzLevdunXzextfffWVZb1Lly62+0hPT7esR0dH224DAAAACEWcMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA4L+ATTo0aN0ujRo72WNWzYUJs2bQr0rhAgx48ft6z/8ccfQR9D48aN/d7GfffdZ1n/5JNPbLdx4403WtYnTpxoWa9QoYLtPuAMsglF8euvv9quY5cLOTk5lvWNGzfa7mPp0qW261hp2rSp7TqjRo2yrN90001+jQH5kUsAzhTwxkw69Z/sxYsX/28n5YKyGwDwCdkEINSQSwBOC8q7v1y5coqNjQ3GpgGgyMgmAKGGXAJwWlA+Y7ZlyxbFxcWpXr16uu222/Tbb7+dc92cnBxlZ2d7PQAgGMgmAKHGl1ySyCagJAt4Y9ayZUtNmzZNCxcu1OTJk7Vjxw5dffXVOnjwYIHrjx07VhEREZ5HQkJCoIcEAGQTgJDjay5JZBNQkgW8MUtJSVHv3r116aWXKjk5WZ999pkyMzP10UcfFbj+iBEjlJWV5XmkpaUFekgAQDYBCDm+5pJENgElWdA/YRoZGakGDRpo69atBdbdbrfcbnewhwEAXsgmAKHGLpcksgkoyYI+j9mhQ4e0bds21axZM9i7AoBCI5sAhBpyCSjdAn7G7NFHH9UNN9ygOnXqaPfu3Ro5cqTKli2rW265JdC7QgkydOhQ23W++OILy/rbb7/t9zjstlG9enXL+gsvvOD3GBAcZFPp9Msvv1jWJ02aZFl/7733bPeRlZXl05iKIjEx0bK+c+dOy/qGDRts93H77bdb1pctW2a7jZYtW9qug/8hlwCcKeCN2a5du3TLLbdo//79io6OVrt27bRq1SpFR0cHelcAUGhkE4BQQy4BOFPAG7NZs2YFepMA4DeyCUCoIZcAnCnonzEDAAAAAFijMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgsIDflREoik8++cR2ndWrV5+HkVjbvn2700MASgxjjGU9NzfXsv7iiy/a7mPChAmWdbs5yG677TbbfcTExFjWO3fubFk/evSo7T7atm1rWf/yyy8t64V5HceOHbOsv/zyy7bb+PDDD23XAQAUjDNmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYUwwDVWtWtWyftlll9lu44cffrCs200k++mnn9ru43ywG+dHH31kWX/ttdds9xEdHe3TmICS6j//+Y9l/YorrvB7HwkJCZb1d955x7Leo0cPv8dwPthNch0I2dnZQd8HAJRmnDEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHMY8ZlBYWJhl/ZFHHrHdxh133OHXGFwul+06brfbsj5o0CDL+vz58233sXPnTsu63Tg/+eQT233cfffdtusAxd20adNs17n//vv92kdh3ksjRoywrCcmJvo1hlCxfft2p4cA+C0jI8OvuiQ1adLEsl6mDOckELr46QQAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA7zeYLplStXavz48Vq3bp327NmjefPmqXv37p66MUYjR47UW2+9pczMTLVt21aTJ09W/fr1AzlunEe9evWyXWf8+PGW9f/85z9+j6Nt27aW9RdffNGy/vnnn/s9BjvJyclB3wfyI5cCyxhju86jjz5qWX/99ddtt1G2bFnL+t/+9jfL+uDBg233YTcxfUkRExMT9H30798/6Psoacgm39xxxx2W9UWLFtluIyoqyrLerFkz22107drVsn7bbbdZ1mNjY233ARTE5zNmhw8fVrNmzTRp0qQC6+PGjdOrr76qKVOmaPXq1apcubKSk5N17NgxvwcLAAUhlwCEIrIJgC98PmOWkpKilJSUAmvGGE2YMEF//etf1a1bN0nSe++9p5iYGM2fP199+/b1b7QAUAByCUAoIpsA+CKgnzHbsWOH0tPT1alTJ8+yiIgItWzZUqmpqQU+JycnR9nZ2V4PAAiUouSSRDYBCC6yCcDZAtqYpaenS8p/rXtMTIyndraxY8cqIiLC80hISAjkkACUckXJJYlsAhBcZBOAszl+V8YRI0YoKyvL80hLS3N6SABANgEISWQTUHIFtDE7fReajIwMr+UZGRnnvEON2+1WeHi41wMAAqUouSSRTQCCi2wCcLaANmaJiYmKjY3VkiVLPMuys7O1evVqtW7dOpC7AoBCIZcAhCKyCcDZfL4r46FDh7R161bP1zt27NCPP/6oqKgo1a5dW0OHDtVzzz2n+vXrKzExUU8//bTi4uK85u1A8VKhQgXbdebMmWNZ79Chg2V99+7dtvv45ptvLOtdunSxrP/yyy+2+7BTo0YNy3rVqlX93gd8Ry4F1vTp023XeeWVVyzrERERttuYP3++ZT0pKcl2Gzhl9erVfm/Dbs63uLg4v/dR2pBNvrH7PX7hhRfabuPXX3+1rG/fvt12G4899phl/f3337esf/bZZ7b7qFmzpu06KH18bszWrl2rjh07er4ePny4JKlfv36aNm2aHn/8cR0+fFj33nuvMjMz1a5dOy1cuFAVK1YM3KgB4AzkEoBQRDYB8IXPjVmHDh1kjDln3eVyacyYMRozZoxfAwOAwiKXAIQisgmALxy/KyMAAAAAlHY0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAc5vNdGYGC2M0t0q1bN8v65MmTbfeRk5NjWV+xYoXtNvxlN87IyMigjwHwl928gXZz+BTGoEGDbNdhnrLC++OPPyzrb7zxht/7eOihhyzrV199td/7AKycnk7AaRs2bLCsX3XVVZb12bNn2+7D7v2G0okzZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGFMMI2QYIxxegiSJLfbbVmvVavWeRoJEDzr16+3rF9yySW224iKirKs33vvvbbbyMrKsqzn5eXZbsPOmjVrLOvh4eGW9YYNG/o9hsqVK1vWK1SoYLuNJUuWWNb3799vWW/Tpo3tPrp162a7DlAaHD161LJ+4sQJy3pOTk4gh4NShDNmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMOYxwwB8be//c2y/sknn1jWXS5XIIdTZK+//rpl/aqrrjpPIwGCp0uXLn7VC2PBggW26/Tt29eybpcLTZo08WlMBTl06JBlfePGjbbbaNasmWX9yiuvtKzffvvttvu48847bdexMmXKFNt1AnE8gVB3/Phx23XsMjAsLMyy3rVrV5/GFCy7d++2rNvNx1aSREREWNYrVqxoWbeb5zZQOGMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOMznecxWrlyp8ePHa926ddqzZ4/mzZun7t27e+r9+/fXu+++6/Wc5ORkLVy40O/BIjgOHz5sWZ84caLtNp555hnLem5urmX9fMxjNnDgQNt1BgwYEPRxIPDIpfNv7NixlvWXXnrJdht2ufD+++9b1nv37m27Dzt285j99NNPttu49NJLLevz58+3rG/bts12H6VpvqGShGwKPU8//bTtOgcOHLCs16pVy7L+yiuv+DSmgnzzzTeW9ZycHNttpKWlWdbtMrgkueCCCyzrdjm+fPnyAI7m3Hw+Y3b48GE1a9ZMkyZNOuc6Xbp00Z49ezyPDz74wK9BAoAVcglAKCKbAPjC5zNmKSkpSklJsVzH7XYrNja2yIMCAF+QSwBCEdkEwBdB+YzZ8uXLVaNGDTVs2FD333+/9u/fH4zdAEChkUsAQhHZBOA0n8+Y2enSpYtuuukmJSYmatu2bXrqqaeUkpKi1NRUlS1bNt/6OTk5XtfJZmdnB3pIAEo5X3NJIpsABB/ZBOBMAW/M+vbt6/l306ZNdemll+rCCy/U8uXLde211+Zbf+zYsRo9enSghwEAHr7mkkQ2AQg+sgnAmYJ+u/x69eqpevXq2rp1a4H1ESNGKCsry/Owu4MMAPjLLpcksgnA+Uc2AaVbwM+YnW3Xrl3av3+/atasWWDd7XbL7XYHexgA4GGXSxLZBOD8I5uA0s3nxuzQoUNef8nZsWOHfvzxR0VFRSkqKkqjR49Wz549FRsbq23btunxxx/XRRddpOTk5IAOHABOI5cAhCKyCYAvfG7M1q5dq44dO3q+Hj58uCSpX79+mjx5stavX693331XmZmZiouLU+fOnfXss8/y1x0HzZkzx7Jud616YSZYPR/sJnT8/fffLesVK1YM5HAQQsilwEpPT7ddZ8yYMZb1pk2b2m7DbuLluLg42234q0qVKpb1Fi1a+L0Pl8tlWbc7lpIUHh5uWZ8wYYJl/eKLL7bdBwKPbDr/7O5qafdeKQy7/2+8/fbbfu/DTkREhO06UVFRlvVGjRpZ1q3O3J5Wvnx5y3qlSpUs63Xr1rXdh93k0IVxxRVXWNYbN27s9z4CwefGrEOHDjLGnLP+xRdf+DUgAPAVuQQgFJFNAHwR9Jt/AAAAAACs0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4zOfb5eP8+u9//2tZf/7552238dlnn1nWDx8+7NOYiiIpKcmy/pe//MV2GytXrrSs2x2LDRs22O4DgNS6dWvbdcqVs/718cYbb9hu43zMU3Y+ZGVlWdafeOIJy3pOTo7tPoYOHWpZ79+/v+02gNLgxIkTlnW7+awk+/m77Obeqlevnu0+2rVrZ1mvUKGCZb0w+RkZGWm7DkILZ8wAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAhzGPWRDZzaWxbNky22307dvXsn7gwAHbbbhcLtt1/DVr1izLep8+fSzrdvMASdL8+fMt68YYy/r69ett9wGUBnZzAqanp9tuo1u3bpb1yy+/3KcxFWcpKSmW9d9++82yPmDAANt9jBo1ypchAaVWTEyMZT01NfU8jQTwHWfMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw5hgOojWrFljWe/SpYvf+yjM5NF267jdbsv6oEGDbPfRtWtXy/obb7xhWZ84caLtPn766SfLut3rvPXWW233AZQGu3fvtqzn5OTYbqNFixaBGo6jfv75Z8u6XbZJ0s6dOy3rjz76qGX92Weftd0HAKDk44wZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DCf5jEbO3asPv74Y23atElhYWFq06aNXnrpJTVs2NCzzrFjx/TII49o1qxZysnJUXJysl5//XXFxMQEfPCh7ssvv3R6CIWSm5trWZ85c6btNuzW2bt3r2W9MPOx+euqq64K+j7gDLLJNxkZGX5v4/vvvw/ASILPbo6xhx9+2K/nS9J7771nWe/QoYNl3W4uSRRfZBMAX/h0xmzFihUaPHiwVq1apUWLFik3N1edO3fW4cOHPesMGzZMn376qWbPnq0VK1Zo9+7duummmwI+cAA4jWwCEIrIJgC+8OmM2cKFC72+njZtmmrUqKF169apffv2ysrK0jvvvKOZM2fqmmuukSRNnTpVF198sVatWqVWrVoFbuQA8P+RTQBCEdkEwBd+fcYsKytLkhQVFSVJWrdunXJzc9WpUyfPOo0aNVLt2rWVmprqz64AoNDIJgChiGwCYMWnM2ZnysvL09ChQ9W2bVs1adJEkpSenq4KFSooMjLSa92YmBilp6cXuJ2cnBzl5OR4vs7Ozi7qkACAbAIQksgmAHaKfMZs8ODB2rBhg2bNmuXXAMaOHauIiAjPIyEhwa/tASjdyCYAoYhsAmCnSI3ZkCFDtGDBAi1btkzx8fGe5bGxsTp+/LgyMzO91s/IyFBsbGyB2xoxYoSysrI8j7S0tKIMCQDIJgAhiWwCUBg+NWbGGA0ZMkTz5s3T0qVLlZiY6FVv3ry5ypcvryVLlniWbd68Wb/99ptat25d4DbdbrfCw8O9HgDgC7IJQCgimwD4wqfPmA0ePFgzZ87UJ598oqpVq3quf46IiFBYWJgiIiJ01113afjw4YqKilJ4eLgefPBBtW7dmjsLhbC8vDzL+h9//HGeRmKtTp06lvX77rvPst69e/cAjgahhGzyzerVq/3exoEDByzru3fvtt1GXFycX2MYP3687Tp///vfLet2cyi+/PLLtvu45ZZbLOtly5a13QZKJrIJgC98aswmT54sKf9kmVOnTlX//v0lSa+88orKlCmjnj17ek2UCADBQjYBCEVkEwBf+NSYGWNs16lYsaImTZqkSZMmFXlQAOALsglAKCKbAPjCr3nMAAAAAAD+ozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwmE+3y4dvBgwYYFmfOXOm7Ta2bt0aqOE4KikpybL+l7/8xXYbzZo1s6xHR0f7NCagpMrNzbWs//77737vY+HChZb1FStW2G6jfPnylvVPP/3Usv7+++/b7uPWW2+1rE+dOtWyXq4cvyYBAOcHZ8wAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAhzFBSxDVrl3bsr558+bzNBIApUlGRoZl/eeffw76GG677Ta/t9GkSRPL+htvvGG7jTvuuMOyzjxlAIBQwRkzAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMGbWBIASJj4+3rL+8ssvW9bvvPNO231Uq1bNsj527FjbbTRv3tyyXq9ePct6RESE7T4AACguOGMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOMyneczGjh2rjz/+WJs2bVJYWJjatGmjl156SQ0bNvSs06FDB61YscLreffdd5+mTJkSmBEDwFnIJt/cfvvtftUBFA7ZBMAXPp0xW7FihQYPHqxVq1Zp0aJFys3NVefOnXX48GGv9e655x7t2bPH8xg3blxABw0AZyKbAIQisgmAL3w6Y7Zw4UKvr6dNm6YaNWpo3bp1at++vWd5pUqVFBsbG5gRAoANsglAKCKbAPjCr8+YZWVlSZKioqK8ls+YMUPVq1dXkyZNNGLECB05cuSc28jJyVF2drbXAwD8QTYBCEVkEwArPp0xO1NeXp6GDh2qtm3bqkmTJp7lt956q+rUqaO4uDitX79eTzzxhDZv3qyPP/64wO2MHTtWo0ePLuowAMAL2QQgFJFNAOy4jDGmKE+8//779fnnn+vrr79WfHz8OddbunSprr32Wm3dulUXXnhhvnpOTo5ycnI8X2dnZyshIUFZWVkKDw8vytAABFl2drYiIiJC8n1KNgGlF9kUWq8ZwCmFzaYinTEbMmSIFixYoJUrV1qGiyS1bNlSks4ZMG63W263uyjDAAAvZBOAUEQ2ASgMnxozY4wefPBBzZs3T8uXL1diYqLtc3788UdJUs2aNYs0QACwQzYBCEVkEwBf+NSYDR48WDNnztQnn3yiqlWrKj09XZIUERGhsLAwbdu2TTNnztR1112natWqaf369Ro2bJjat2+vSy+9NCgvAADIJgChiGwC4AufPmPmcrkKXD516lT1799faWlpuv3227VhwwYdPnxYCQkJ6tGjh/76178W+rrnUL4+HMApofY+JZsASKH3PiWbAEhB+oyZXQ+XkJCQb/Z6AAg2sglAKCKbAPjCr3nMAAAAAAD+ozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4LByTg/gbMYYSVJ2drbDIwFwLqffn6ffr6UB2QSEPrIJQCgqbDaFXGN28OBBSVJCQoLDIwFg5+DBg4qIiHB6GOcF2QQUH2QTgFBkl00uE2J/VsrLy9Pu3btVtWpVuVwuSae6zISEBKWlpSk8PNzhERZvHMvAKq3H0xijgwcPKi4uTmXKlI4ros/OptL6vQ8WjmfglOZjSTaRTYHG8Qyc0nwsC5tNIXfGrEyZMoqPjy+wFh4eXuq+kcHCsQys0ng8S8tfo087VzaVxu99MHE8A6e0Hkuy6ZTS+v0PFo5n4JTWY1mYbCodf04CAAAAgBBGYwYAAAAADisWjZnb7dbIkSPldrudHkqxx7EMLI5n6cX3PrA4noHDsSzd+P4HFsczcDiW9kLu5h8AAAAAUNoUizNmAAAAAFCS0ZgBAAAAgMNozAAAAADAYTRmAAAAAOCwkG/MJk2apLp166pixYpq2bKlvvvuO6eHVCysXLlSN9xwg+Li4uRyuTR//nyvujFGzzzzjGrWrKmwsDB16tRJW7ZscWawIW7s2LG66qqrVLVqVdWoUUPdu3fX5s2bvdY5duyYBg8erGrVqqlKlSrq2bOnMjIyHBoxzgeyqWjIpsAhm1AQssl35FLgkEv+CenG7MMPP9Tw4cM1cuRIff/992rWrJmSk5O1d+9ep4cW8g4fPqxmzZpp0qRJBdbHjRunV199VVOmTNHq1atVuXJlJScn69ixY+d5pKFvxYoVGjx4sFatWqVFixYpNzdXnTt31uHDhz3rDBs2TJ9++qlmz56tFStWaPfu3brpppscHDWCiWwqOrIpcMgmnI1sKhpyKXDIJT+ZENaiRQszePBgz9cnT540cXFxZuzYsQ6OqviRZObNm+f5Oi8vz8TGxprx48d7lmVmZhq3220++OADB0ZYvOzdu9dIMitWrDDGnDp25cuXN7Nnz/as8/PPPxtJJjU11alhIojIpsAgmwKLbALZ5D9yKbDIJd+E7Bmz48ePa926derUqZNnWZkyZdSpUyelpqY6OLLib8eOHUpPT/c6thEREWrZsiXHthCysrIkSVFRUZKkdevWKTc31+t4NmrUSLVr1+Z4lkBkU/CQTf4hm0o3sik4yCX/kEu+CdnGbN++fTp58qRiYmK8lsfExCg9Pd2hUZUMp48fx9Z3eXl5Gjp0qNq2basmTZpIOnU8K1SooMjISK91OZ4lE9kUPGRT0ZFNIJuCg1wqOnLJd+WcHgBQnAwePFgbNmzQ119/7fRQAMCDbAIQasgl34XsGbPq1aurbNmy+e7SkpGRodjYWIdGVTKcPn4cW98MGTJECxYs0LJlyxQfH+9ZHhsbq+PHjyszM9NrfY5nyUQ2BQ/ZVDRkEySyKVjIpaIhl4omZBuzChUqqHnz5lqyZIlnWV5enpYsWaLWrVs7OLLiLzExUbGxsV7HNjs7W6tXr+bYFsAYoyFDhmjevHlaunSpEhMTverNmzdX+fLlvY7n5s2b9dtvv3E8SyCyKXjIJt+QTTgT2RQc5JJvyCU/OXzzEUuzZs0ybrfbTJs2zfz000/m3nvvNZGRkSY9Pd3poYW8gwcPmh9++MH88MMPRpJ5+eWXzQ8//GB+/fVXY4wxL774oomMjDSffPKJWb9+venWrZtJTEw0R48edXjkoef+++83ERERZvny5WbPnj2ex5EjRzzrDBo0yNSuXdssXbrUrF271rRu3dq0bt3awVEjmMimoiObAodswtnIpqIhlwKHXPJPSDdmxhjz2muvmdq1a5sKFSqYFi1amFWrVjk9pGJh2bJlRlK+R79+/Ywxp27/+vTTT5uYmBjjdrvNtddeazZv3uzsoENUQcdRkpk6dapnnaNHj5oHHnjAXHDBBaZSpUqmR48eZs+ePc4NGkFHNhUN2RQ4ZBMKQjb5jlwKHHLJPy5jjAnuOTkAAAAAgJWQ/YwZAAAAAJQWNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGbFxLRp0+RyubRz586Ab7tu3brq379/wLdbWP3791fdunW9lh06dEh33323YmNj5XK5NHToUO3cuVMul0vTpk0L2L6DeVyB0oBsIpuAUEQ2kU3FEY1ZIfXv318ul+ucj99//93pIZYoL7zwgqZNm6b7779f06dP1x133OH0kIJu3bp16tKli8LDw1W1alV17txZP/74o9PDQogjm86v0pZNa9as0ZAhQ9S4cWNVrlxZtWvXVp8+ffTLL784PTSEOLLp/CKbSkY2uYwxxulBFAepqanatm2b1zJjjAYNGqS6detq48aNQd3/yZMnlZubK7fbLZfLFdBt161bVx06dAjoX1R8kZubq7y8PLndbs+yVq1aqVy5cvr66689y4wxysnJUfny5VW2bNmA7HvatGkaMGCAduzYke+vT+fT999/r7Zt2yohIUH33Xef8vLy9Prrr+vAgQP67rvv1LBhQ8fGhtBGNgUP2ST16tVL33zzjXr37q1LL71U6enpmjhxog4dOqRVq1apSZMmjo0NoY1sCh6yqeRmUzmnB1BctG7dWq1bt/Za9vXXX+vIkSO67bbbgr7/smXLBuxNFWrKly+fb9nevXt1ySWXeC1zuVyqWLHi+RrWefX0008rLCxMqampqlatmiTp9ttvV4MGDfTUU09p7ty5Do8QoYpsCh6ySRo+fLhmzpypChUqeJbdfPPNatq0qV588UW9//77Do4OoYxsCh6yqeRmE5cy+mHmzJlyuVy69dZbbdft0KGDmjRpovXr1yspKUmVKlXSRRddpDlz5kiSVqxYoZYtWyosLEwNGzbU4sWLvZ5f0DW9a9euVXJysqpXr66wsDAlJiZq4MCBXs/Ly8vTP/7xDzVt2lQVK1ZUdHS0unTporVr155zrAcOHNCjjz6qpk2bqkqVKgoPD1dKSor+85//5Fv3tddeU+PGjVWpUiVdcMEFuvLKKzVz5kxP/eDBgxo6dKjq1q0rt9utGjVq6P/+7//0/fffe9Y581rp5cuXy+VyaceOHfr3v//tueRh586d57xWetOmTerVq5eioqJUsWJFXXnllfrXv/6Vb6wbN27UNddco7CwMMXHx+u5555TXl7eOY/DmdavX6/+/furXr16qlixomJjYzVw4EDt37/fa73CvN6CfPXVV+rUqZOnKZOkmjVrKikpSQsWLNChQ4cKNU5AIpsksilQ2dSmTRuv//hIUv369dW4cWP9/PPPhRojcBrZRDaRTdY4Y1ZEubm5+uijj9SmTZtCn8r9888/df3116tv377q3bu3Jk+erL59+2rGjBkaOnSoBg0apFtvvVXjx49Xr169lJaWpqpVqxa4rb1796pz586Kjo7Wk08+qcjISO3cuVMff/yx13p33XWXpk2bppSUFN199906ceKEvvrqK61atUpXXnllgdvevn275s+fr969eysxMVEZGRl64403lJSUpJ9++klxcXGSpLfeeksPPfSQevXqpYcffljHjh3T+vXrtXr1ak/oDho0SHPmzNGQIUN0ySWXaP/+/fr666/1888/64orrsi374svvljTp0/XsGHDFB8fr0ceeUSSFB0drT/++CPf+hs3blTbtm1Vq1YtPfnkk6pcubI++ugjde/eXXPnzlWPHj0kSenp6erYsaNOnDjhWe/NN99UWFhYob53ixYt0vbt2zVgwADFxsZq48aNevPNN7Vx40atWrXKc5mEr6/3tJycnALHUqlSJR0/flwbNmxQq1atCjVWlG5kE9kUyGwqiDFGGRkZaty4sU/PQ+lGNpFNZFMhGBTJp59+aiSZ119/vVDrJyUlGUlm5syZnmWbNm0ykkyZMmXMqlWrPMu/+OILI8lMnTrVs2zq1KlGktmxY4cxxph58+YZSWbNmjXn3OfSpUuNJPPQQw/lq+Xl5Xn+XadOHdOvXz/P18eOHTMnT570Wn/Hjh3G7XabMWPGeJZ169bNNG7c2PJ1R0REmMGDB1uu069fP1OnTh2vZXXq1DFdu3bNN4azj8u1115rmjZtao4dO+b12tq0aWPq16/vWTZ06FAjyaxevdqzbO/evSYiIsLruJ7LkSNH8i374IMPjCSzcuVKz7LCvN6CNG3a1DRo0MCcOHHCsywnJ8fUrl3bSDJz5szxeZsoncgmsimQ2VSQ6dOnG0nmnXfeCcj2UDqQTWQT2WSPSxmLaObMmSpfvrz69OlT6OdUqVJFffv29XzdsGFDRUZG6uKLL1bLli09y0//e/v27efcVmRkpCRpwYIFys3NLXCduXPnyuVyaeTIkflqVh+EdbvdKlPm1I/GyZMntX//flWpUkUNGzb0OrUcGRmpXbt2ac2aNZbjXL16tXbv3n3OdYrqwIEDWrp0qfr06aODBw9q37592rdvn/bv36/k5GRt2bLFc9enzz77TK1atVKLFi08z4+Oji70de5n/oXo2LFj2rdvn+cM1tnHpCiv94EHHtAvv/yiu+66Sz/99JM2bNigO++8U3v27JEkHT161KftofQim8imQGbT2TZt2qTBgwerdevW6tevn1/bQulCNpFNZJM9GrMiOHTokD755BMlJyd7fSbITnx8fL43dkREhBISEvItk06dwj+XpKQk9ezZU6NHj1b16tXVrVs3TZ06VTk5OZ51tm3bpri4OEVFRRV6jNKp66tfeeUV1a9fX263W9WrV1d0dLTWr1+vrKwsz3pPPPGEqlSpohYtWqh+/foaPHiwvvnmG69tjRs3Ths2bFBCQoJatGihUaNGWQanL7Zu3SpjjJ5++mlFR0d7PU6H6t69eyVJv/76q+rXr59vG4W92+GBAwf08MMPKyYmRmFhYYqOjlZiYqIkeR2Tor7eQYMG6amnntLMmTPVuHFjNW3aVNu2bdPjjz8u6dQvJ8AO2XQK2RS4bDpTenq6unbtqoiICM2ZM6fE3lgBgUc2nUI2kU12aMyKYP78+UW6q9C5flDOtdxYzGTgcrk0Z84cpaamasiQIfr99981cOBANW/e3O8bRbzwwgsaPny42rdvr/fff19ffPGFFi1apMaNG3t96PPiiy/W5s2bNWvWLLVr105z585Vu3btvP7S1KdPH23fvl2vvfaa4uLiNH78eDVu3Fiff/65X2OU5BnLo48+qkWLFhX4uOiii/zej3Tqdbz11lsaNGiQPv74Y3355ZdauHCh1zhOr1fU1/v8888rIyNDX331ldavX681a9Z4tt2gQYOAvA6UbGTTKWRTYLNJOvUfqZSUFGVmZmrhwoWez8wAhUE2nUI2kU22nLyOsrjq0qWLqVKlijl8+HChn5OUlFTgdcUFXRNsjDGSvK65Pfta6YLMmDHDSDJvvfWWMcaYwYMHG5fLZfbv3285trOvlW7WrJnp2LFjvvVq1aplkpKSzrmdnJwc07VrV1O2bFlz9OjRAtfJyMgwtWrVMm3btvUsK+q10hkZGUaSGTFihOXrM8aYBg0amFatWuVb/sADD9ge1wMHDhhJZvTo0V7Lf/nlFyPJjBw58pzPLej1+uKqq64y8fHx+a5dBwpCNhWMbMrPl2w6evSoufrqq02lSpXMt99+a7s+cDayqWBkU36lPZs4Y+ajP/74Q4sXL1aPHj1UqVIlx8bx559/5vvL0GWXXSZJntPyPXv2lDFGo0ePzvf8s597prJly+arz54923Pd8Wln3/K0QoUKuuSSS2SMUW5urk6ePOl1ulqSatSoobi4OK9LB4qqRo0a6tChg9544w3PZ7HOdObdiK677jqtWrVK3333nVd9xowZtvs5/Ze5s4/JhAkTvL4O9Ov98MMPtWbNGg0dOtRz7TpwLmTT/5BNE7y+9uf1njx5UjfffLNSU1M1e/bsfPNSAXbIpv8hmyZ4fU025cft8n304Ycf6sSJE+dlckQr7777rl5//XX16NFDF154oQ4ePKi33npL4eHhuu666yRJHTt21B133KFXX31VW7ZsUZcuXZSXl6evvvpKHTt21JAhQwrc9vXXX68xY8ZowIABatOmjf773/9qxowZqlevntd6nTt3VmxsrNq2bauYmBj9/PPPmjhxorp27aqqVasqMzNT8fHx6tWrl5o1a6YqVapo8eLFWrNmjf7+978H5DhMmjRJ7dq1U9OmTXXPPfeoXr16ysjIUGpqqnbt2uWZQ+Txxx/X9OnT1aVLFz388MOe277WqVNH69evt9xHeHi42rdvr3Hjxik3N1e1atXSl19+qR07dnitd/DgwSK/3pUrV2rMmDHq3LmzqlWrplWrVmnq1Kme8QJ2yKb/IZsCl02PPPKI/vWvf+mGG27QgQMH8k3aevvttxfh6KA0IZv+h2wim2yd71N0xV2rVq1MjRo1vG5rXhiBPiX//fffm1tuucXUrl3buN1uU6NGDXP99debtWvXem3nxIkTZvz48aZRo0amQoUKJjo62qSkpJh169Z5jeHs274+8sgjpmbNmiYsLMy0bdvWpKammqSkJK9T8m+88YZp3769qVatmnG73ebCCy80jz32mMnKyjLGnDpF/9hjj5lmzZqZqlWrmsqVK5tmzZrlu1WuP7d9NcaYbdu2mTvvvNPExsaa8uXLm1q1apnrr78+3y3m169fb5KSkkzFihVNrVq1zLPPPmveeeedQt32ddeuXaZHjx4mMjLSREREmN69e5vdu3d7nZIv7OstyNatW03nzp1N9erVjdvtNo0aNTJjx441OTk5ts8FjCGbyKbgZNPpW5af6wHYIZuSPOuRTWSTHZcxFudmAQAAAABBxwdXAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOC7kJpvPy8rR7925VrVpVLpfL6eEAKIAxRgcPHlRcXJzKlCkdf98hm4DQRzaRTUAoKnQ2BWuCtIkTJ5o6deoYt9ttWrRoYVavXl2o56WlpVlOGMeDB4/QeaSlpQUrQoKGbOLBo+Q/ils2FTWXjCGbePAoTg+7bArKGbMPP/xQw4cP15QpU9SyZUtNmDBBycnJ2rx5s2rUqGH53KpVq0qS0tLSFB4eHozhAfBTdna2EhISPO/X4oJsAkq24phN/uSSRDYBxUFhs8lljDGB3nnLli111VVXaeLEiZJOnWZPSEjQgw8+qCeffNLyudnZ2YqIiFBWVhYBA4So4vo+JZuAkq04vk/9ySWpeL5moLQp7Ps04BdgHz9+XOvWrVOnTp3+t5MyZdSpUyelpqYGencAUChkE4BQQy4BOFPAL2Xct2+fTp48qZiYGK/lMTEx2rRpU771c3JylJOT4/k6Ozs70EMCALIJQMjxNZcksgkoyRy/ZdHYsWMVERHheSQkJDg9JAAgmwCEJLIJKLkC3phVr15dZcuWVUZGhtfyjIwMxcbG5lt/xIgRysrK8jzS0tICPSQAIJsAhBxfc0kim4CSLOCNWYUKFdS8eXMtWbLEsywvL09LlixR69at863vdrsVHh7u9QCAQCObAIQaX3NJIpuAkiwot8sfPny4+vXrpyuvvFItWrTQhAkTdPjwYQ0YMCAYuwOAQiGbAIQacgnAaUFpzG6++Wb98ccfeuaZZ5Senq7LLrtMCxcuzPfhVgA4n8gmAKGGXAJwWlDmMfMH83EAoa80vk9L42sGipvS+D4tja8ZKG4cm8cMAAAAAOAbGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMPKBXqDo0aN0ujRo72WNWzYUJs2bQr0rhBCfvzxR8v6mDFjLOvz5s2z3UenTp0s6++++65lPS4uznYfKLnIJgChhlwqmfr06eP3Nj766KMAjATFTcAbM0lq3LixFi9e/L+dlAvKbgDAJ2QTgFBDLgE4LSjv/nLlyik2NjYYmwaAIiObAIQacgnAaUH5jNmWLVsUFxenevXq6bbbbtNvv/12znVzcnKUnZ3t9QCAYCCbAIQaX3JJIpuAkizgjVnLli01bdo0LVy4UJMnT9aOHTt09dVX6+DBgwWuP3bsWEVERHgeCQkJgR4SAJBNAEKOr7kkkU1ASeYyxphg7iAzM1N16tTRyy+/rLvuuitfPScnRzk5OZ6vs7OzlZCQoKysLIWHhwdzaAggbv5RumRnZysiIqJYv0/JJqDkKe7ZZJdLEtlUHHDzD5ytsNkU9E+YRkZGqkGDBtq6dWuBdbfbLbfbHexhAIAXsglAqLHLJYlsAkqyoM9jdujQIW3btk01a9YM9q4AoNDIJgChhlwCSreAnzF79NFHdcMNN6hOnTravXu3Ro4cqbJly+qWW24J9K5wnkycONF2nQcffNCyHhUVZVm3u0xRkuVfECWpSZMmlvVVq1bZ7qNBgwa266B4Kk3ZtHz5cst6x44dz89AQkCHDh0s60lJSZb1UaNGBW4wwFlKUy6VJKmpqZb1wvx/Iy0tzbJudzkklzqWTAFvzHbt2qVbbrlF+/fvV3R0tNq1a6dVq1YpOjo60LsCgEIjmwCEGnIJwJkC3pjNmjUr0JsEAL+RTQBCDbkE4ExB/4wZAAAAAMAajRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAhwX8rowofpYtW2ZZf/TRR223ER4eblmfPXu2Zf2aa66x3cdPP/1kWbebx+zrr7+23QfzmCHU2c1RJpWuecrs2B0vu/qKFSts98FcaADO9Le//c12Hbu5zl555RXLemHmMbObCw2hhzNmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYUwwXQqcOHHCsv7pp59a1suVs/8x+fzzzy3rbdq0sd2GnejoaMu62+22rA8aNMh2HzfccINfYwCKgw4dOljW7SZMLk7sJoj2dwLqwqwzevRoy/qyZcts92H3PQNw/rRu3dqveiA8+uijtuvYjSMhISFQw0GAcMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBjzmJUCdnPkTJgwwbI+duxY230EYp4yO3ZziPXo0cOyPmvWLNt9GGN8GhNwvhVmPiu79zxzYv2Pv3OQFWYbdjp27Gi7zsiRIy3ro0aN8msMAM6vPn36WNZ37dplWbf7v5sktW3b1rLeq1cvy/rLL79suw8EFmfMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw3yeYHrlypUaP3681q1bpz179mjevHnq3r27p26M0ciRI/XWW28pMzNTbdu21eTJk1W/fv1Ajhs++Oijj/x6vt0kiIDTyCVvTCBdeHbHqjDH0t9JqgszQbXdNgLxOhB4ZBOKavjw4X7VJal169aW9VdeecWynpqaaruPwqyDwvP5jNnhw4fVrFkzTZo0qcD6uHHj9Oqrr2rKlClavXq1KleurOTkZB07dszvwQJAQcglAKGIbALgC5/PmKWkpCglJaXAmjFGEyZM0F//+ld169ZNkvTee+8pJiZG8+fPV9++ff0bLQAUgFwCEIrIJgC+COhnzHbs2KH09HR16tTJsywiIkItW7Y856nOnJwcZWdnez0AIFCKkksS2QQguMgmAGcLaGOWnp4uSYqJifFaHhMT46mdbezYsYqIiPA8EhISAjkkAKVcUXJJIpsABBfZBOBsjt+VccSIEcrKyvI80tLSnB4SAJBNAEIS2QSUXAFtzGJjYyVJGRkZXsszMjI8tbO53W6Fh4d7PQAgUIqSSxLZBCC4yCYAZwtoY5aYmKjY2FgtWbLEsyw7O1urV6+2vWUnAAQDuQQgFJFNAM7m810ZDx06pK1bt3q+3rFjh3788UdFRUWpdu3aGjp0qJ577jnVr19fiYmJevrppxUXF+c1bwdCy5kfPC5IfHz8eRqJNWOMZf3EiROW9caNG9vuo3Llyj6NCaGBXIKT/J1DzOVy+T2Gjh07Wtbt8hPBQTbBSXZzjNnNUzt79mzbfdjNp/byyy/bbgP/43NjtnbtWq9fAKe/If369dO0adP0+OOP6/Dhw7r33nuVmZmpdu3aaeHChapYsWLgRg0AZyCXAIQisgmAL3xuzDp06GD5lzeXy6UxY8ZozJgxfg0MAAqLXAIQisgmAL5w/K6MAAAAAFDa0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcJjPd2VE8VOrVi3L+r59+yzrx44ds91H+fLlfRpTUezdu9eyPmfOHMt63759bffBPGYAAKA0+OijjyzrhZlj8ZVXXrGsM4+ZbzhjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMxgwAAAAAHMYE06XAqFGjnB5CQGzfvt2v5/fs2TNAIwGA0DJy5EinhwAA8BNnzAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACHMY8Zig1/5yGLj48P0EgAoPACMZdkhw4dgr4PAKVL7dq1/d5G7969AzASnMYZMwAAAABwGI0ZAAAAADiMxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhPs9jtnLlSo0fP17r1q3Tnj17NG/ePHXv3t1T79+/v959912v5yQnJ2vhwoV+Dxb55eXl2a6zc+fOoI8jOzvbsj5t2jTL+saNG233sWfPHl+GlM/u3btt1zlw4IBlPSoqyq8xIDjIJThp+fLllvXRo0f7vY+kpCS/t4Hzj2yCk/r06WNZT0tLs6wPGzbMdh8vv/yyT2OCNZ/PmB0+fFjNmjXTpEmTzrlOly5dtGfPHs/jgw8+8GuQAGCFXAIQisgmAL7w+YxZSkqKUlJSLNdxu92KjY0t8qAAwBfkEoBQRDYB8EVQPmO2fPly1ahRQw0bNtT999+v/fv3B2M3AFBo5BKAUEQ2ATjN5zNmdrp06aKbbrpJiYmJ2rZtm5566imlpKQoNTVVZcuWzbd+Tk6OcnJyPF/bfVYJAHzlay5JZBOA4CObAJwp4I1Z3759Pf9u2rSpLr30Ul144YVavny5rr322nzrjx07NiAfjAaAc/E1lySyCUDwkU0AzhT02+XXq1dP1atX19atWwusjxgxQllZWZ6H3R1iAMBfdrkkkU0Azj+yCSjdAn7G7Gy7du3S/v37VbNmzQLrbrdbbrc72MMAAA+7XJLIJgDnH9kElG4+N2aHDh3y+kvOjh079OOPPyoqKkpRUVEaPXq0evbsqdjYWG3btk2PP/64LrroIiUnJwd04ABwGrkEIBSRTQB84XNjtnbtWnXs2NHz9fDhwyVJ/fr10+TJk7V+/Xq9++67yszMVFxcnDp37qxnn32Wv+4UICsry3adtWvXWtanTJliu425c+cWekxFZYyxrLtcLr/3YbeNiIgIy/rrr79uu4+6deta1plgOjSRS3DSmT97wdKhQ4eg7wOBRzahqFJTUy3rN998s+02/J1Amsmjzz+fG7MOHTpY/if8iy++8GtAAOArcglAKCKbAPgi6Df/AAAAAABYozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwmM+3y0fh/eMf/7Csv/rqq7bb2LFjh9/juOCCCyzrPXv29HsfdhYtWmRZ//XXX223Ua1aNcv66tWrLev16tWz3QcAnOl8zFG2bNky23WYxwwoWU7PaXcur7zyimU9ISHBdh8ffvihZb1Pnz6228D5xRkzAAAAAHAYjRkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGHMYxZE999/v2U9Pj7edhu//PKLZf2ee+6x3UblypUt62FhYbbbsHPy5EnLeteuXS3rhZnH7IMPPrCsM08ZAF8tX77cr3ph2M1BxhxlQPHy8ssvW9YnTJhgu420tDTL+rBhw/yqS4Wb6wyhhTNmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYUwwHUQVKlSwrPfs2fM8jST4Ro8ebVn/8ssvLesdO3a03QeTsALwld0E0YXJHjt22bRs2TK/9wHg/Bk+fLhl/ZVXXrGsF2Zi52+//day3rp1a9ttoOThjBkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMJ/mMRs7dqw+/vhjbdq0SWFhYWrTpo1eeuklNWzY0LPOsWPH9Mgjj2jWrFnKyclRcnKyXn/9dcXExAR88Dg/Nm/ebLvOs88+a1l3uVyW9aeeesp2H+XKMe0eCkY24Vzs5lgMhJEjRwZ9HyieyKbzLy0tzbLetm1bv7fRqlUry/pHH31ku4/CzHWG0senM2YrVqzQ4MGDtWrVKi1atEi5ubnq3LmzDh8+7Fln2LBh+vTTTzV79mytWLFCu3fv1k033RTwgQPAaWQTgFBENgHwhU+nIBYuXOj19bRp01SjRg2tW7dO7du3V1ZWlt555x3NnDlT11xzjSRp6tSpuvjii7Vq1SrbvzAAQFGQTQBCEdkEwBd+fcYsKytLkhQVFSVJWrdunXJzc9WpUyfPOo0aNVLt2rWVmprqz64AoNDIJgChiGwCYKXIH9rJy8vT0KFD1bZtWzVp0kSSlJ6ergoVKigyMtJr3ZiYGKWnpxe4nZycHOXk5Hi+zs7OLuqQAIBsAhCSyCYAdop8xmzw4MHasGGDZs2a5dcAxo4dq4iICM+DD0MC8AfZBCAUkU0A7BSpMRsyZIgWLFigZcuWKT4+3rM8NjZWx48fV2Zmptf6GRkZio2NLXBbI0aMUFZWludhdyccADgXsglAKCKbABSGT42ZMUZDhgzRvHnztHTpUiUmJnrVmzdvrvLly2vJkiWeZZs3b9Zvv/2m1q1bF7hNt9ut8PBwrwcA+IJsAhCKyCYAvvDpM2aDBw/WzJkz9cknn6hq1aqe658jIiIUFhamiIgI3XXXXRo+fLiioqIUHh6uBx98UK1bt+bOQiHsxIkTlvUePXr4vY9HHnnEsp6UlOT3PlB6kU2l0/LlywOyjpXCzFHWoUMHv/aBkotsCqzCnB20m6esMNv49ttvLevnapoBf/nUmE2ePFlS/l9CU6dOVf/+/SVJr7zyisqUKaOePXt6TZQIAMFCNgEIRWQTAF/41JgZY2zXqVixoiZNmqRJkyYVeVAA4AuyCUAoIpsA+MKvecwAAAAAAP6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHCYT7fLR8n02WefWdY3bdpku43o6GjL+oMPPmhZL1eOH0UAvvF38ujCGDVqVND3AeAUu8mf7SaPLsw27CaPlphAGs7hjBkAAAAAOIzGDAAAAAAcRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMCaPgrZv3+73Nuzm+qldu7bf+wCAM40ePdrpIRSKXT4W5nUsW7bMst6hQwcfRgSEJrv/KyQkJNhu47fffvN7G4BTOGMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOIzGDAAAAAAcxgTTUPXq1S3rl112me02HnjggQCNBgAKx27SZUnq2LGjX/twuVx+Pb8wCvM6mEAakP72t7/ZrsME0ijOOGMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDaMwAAAAAwGE0ZgAAAADgMBozAAAAAHAYjRkAAAAAOMyneczGjh2rjz/+WJs2bVJYWJjatGmjl156SQ0bNvSs06FDB61YscLreffdd5+mTJkSmBEj4G6//Xa/6oDTyKbSqTBze40cOdKyPnr06KCPozDzlKFkIpt8Y4xxegiAo3w6Y7ZixQoNHjxYq1at0qJFi5Sbm6vOnTvr8OHDXuvdc8892rNnj+cxbty4gA4aAM5ENgEIRWQTAF/4dMZs4cKFXl9PmzZNNWrU0Lp169S+fXvP8kqVKik2NjYwIwQAG2QTgFBENgHwhV+fMcvKypIkRUVFeS2fMWOGqlevriZNmmjEiBE6cuTIObeRk5Oj7OxsrwcA+INsAhCKyCYAVnw6Y3amvLw8DR06VG3btlWTJk08y2+99VbVqVNHcXFxWr9+vZ544glt3rxZH3/8cYHbGTt2bECu8QcAiWwCEJrIJgB2XKaIn7S8//779fnnn+vrr79WfHz8OddbunSprr32Wm3dulUXXnhhvnpOTo5ycnI8X2dnZyshIUFZWVkKDw8vytAABFl2drYiIiJC8n1KNuFMo0aNsqxz84+ShWwKrdcM4JTCZlORzpgNGTJECxYs0MqVKy3DRZJatmwpSecMGLfbLbfbXZRhAIAXsglAKCKbABSGT42ZMUYPPvig5s2bp+XLlysxMdH2OT/++KMkqWbNmkUaIADYIZsAhCKyCYAvfGrMBg8erJkzZ+qTTz5R1apVlZ6eLkmKiIhQWFiYtm3bppkzZ+q6665TtWrVtH79eg0bNkzt27fXpZdeGpQXAABkE87F7lJGuzrgD7IJgC98+oyZy+UqcPnUqVPVv39/paWl6fbbb9eGDRt0+PBhJSQkqEePHvrrX/9a6OueQ/n6cACnhNr7lGwCIIXe+5RsAiAF6TNmdj1cQkJCvtnrASDYyCYAoYhsAuALv+YxAwAAAAD4j8YMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMPKOT2AsxljJEnZ2dkOjwTAuZx+f55+v5YGZBMQ+sgmAKGosNkUco3ZwYMHJUkJCQkOjwSAnYMHDyoiIsLpYZwXZBNQfJBNAEKRXTa5TIj9WSkvL0+7d+9W1apV5XK5JJ3qMhMSEpSWlqbw8HCHR1i8cSwDq7QeT2OMDh48qLi4OJUpUzquiD47m0rr9z5YOJ6BU5qPJdlENgUaxzNwSvOxLGw2hdwZszJlyig+Pr7AWnh4eKn7RgYLxzKwSuPxLC1/jT7tXNlUGr/3wcTxDJzSeizJplNK6/c/WDiegVNaj2Vhsql0/DkJAAAAAEIYjRkAAAAAOKxYNGZut1sjR46U2+12eijFHscysDiepRff+8DieAYOx7J04/sfWBzPwOFY2gu5m38AAAAAQGlTLM6YAQAAAEBJRmMGAAAAAA6jMQMAAAAAh9GYAQAAAIDDQr4xmzRpkurWrauKFSuqZcuW+u6775weUrGwcuVK3XDDDYqLi5PL5dL8+fO96sYYPfPMM6pZs6bCwsLUqVMnbdmyxZnBhrixY8fqqquuUtWqVVWjRg11795dmzdv9lrn2LFjGjx4sKpVq6YqVaqoZ8+eysjIcGjEOB/IpqIhmwKHbEJByCbfkUuBQy75J6Qbsw8//FDDhw/XyJEj9f3336tZs2ZKTk7W3r17nR5ayDt8+LCaNWumSZMmFVgfN26cXn31VU2ZMkWrV69W5cqVlZycrGPHjp3nkYa+FStWaPDgwVq1apUWLVqk3Nxcde7cWYcPH/asM2zYMH366aeaPXu2VqxYod27d+umm25ycNQIJrKp6MimwCGbcDayqWjIpcAhl/xkQliLFi3M4MGDPV+fPHnSxMXFmbFjxzo4quJHkpk3b57n67y8PBMbG2vGjx/vWZaZmWncbrf54IMPHBhh8bJ3714jyaxYscIYc+rYlS9f3syePduzzs8//2wkmdTUVKeGiSAimwKDbAossglkk//IpcAil3wTsmfMjh8/rnXr1qlTp06eZWXKlFGnTp2Umprq4MiKvx07dig9Pd3r2EZERKhly5Yc20LIysqSJEVFRUmS1q1bp9zcXK/j2ahRI9WuXZvjWQKRTcFDNvmHbCrdyKbgIJf8Qy75JmQbs3379unkyZOKiYnxWh4TE6P09HSHRlUynD5+HFvf5eXlaejQoWrbtq2aNGki6dTxrFChgiIjI73W5XiWTGRT8JBNRUc2gWwKDnKp6Mgl35VzegBAcTJ48GBt2LBBX3/9tdNDAQAPsglAqCGXfBeyZ8yqV6+usmXL5rtLS0ZGhmJjYx0aVclw+vhxbH0zZMgQLViwQMuWLVN8fLxneWxsrI4fP67MzEyv9TmeJRPZFDxkU9GQTZDIpmAhl4qGXCqakG3MKlSooObNm2vJkiWeZXl5eVqyZIlat27t4MiKv8TERMXGxnod2+zsbK1evZpjWwBjjIYMGaJ58+Zp6dKlSkxM9Ko3b95c5cuX9zqemzdv1m+//cbxLIHIpuAhm3xDNuFMZFNwkEu+IZf85PDNRyzNmjXLuN1uM23aNPPTTz+Ze++910RGRpr09HSnhxbyDh48aH744Qfzww8/GEnm5ZdfNj/88IP59ddfjTHGvPjiiyYyMtJ88sknZv369aZbt24mMTHRHD161OGRh57777/fREREmOXLl5s9e/Z4HkeOHPGsM2jQIFO7dm2zdOlSs3btWtO6dWvTunVrB0eNYCKbio5sChyyCWcjm4qGXAoccsk/Id2YGWPMa6+9ZmrXrm0qVKhgWrRoYVatWuX0kIqFZcuWGUn5Hv369TPGnLr969NPP21iYmKM2+021157rdm8ebOzgw5RBR1HSWbq1KmedY4ePWoeeOABc8EFF5hKlSqZHj16mD179jg3aAQd2VQ0ZFPgkE0oCNnkO3IpcMgl/7iMMSa45+QAAAAAAFZC9jNmAAAAAFBa0JgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgVE9OmTZPL5dLOnTsDvu26deuqf//+Ad9uYfXv319169b1Wnbo0CHdfffdio2Nlcvl0tChQ7Vz5065XC5NmzYtYPsO5nEFSgOyiWwCQhHZRDYVRzRmhdS/f3+5XK5zPn7//Xenh1iivPDCC5o2bZruv/9+TZ8+XXfccYfTQwqqQ4cOaeTIkerSpYuioqICHqQo2datW6cuXbooPDxcVatWVefOnfXjjz86PawSqbRlE7/74I8tW7aob9++io+PV6VKldSoUSONGTNGR44ccXpoJU5py6azPf/883K5XGrSpInTQ/FLOacHUFzcd9996tSpk9cyY4wGDRqkunXrqlatWkHd/x133KG+ffvK7XYHdT9OeOutt5SXl+e1bOnSpWrVqpVGjhzpWWaM0dGjR1W+fPnzPcSg27dvn8aMGaPatWurWbNmWr58udNDQjHx/fffq127dkpISNDIkSOVl5en119/XUlJSfruu+/UsGHDoO6fbCrZ2eT07z4UX2lpaWrRooUiIiI0ZMgQRUVFKTU1VSNHjtS6dev0ySefBHX/ZFPJzqYz7dq1Sy+88IIqV67s9FD8RmNWSK1bt1br1q29ln399dc6cuSIbrvttqDvv2zZsipbtmzQ9+OEggJj7969uuSSS7yWuVwuVaxY8XwN67yqWbOm9uzZo9jYWK1du1ZXXXWV00NCMfH0008rLCxMqampqlatmiTp9ttvV4MGDfTUU09p7ty5Qd0/2VSys8np330ovqZPn67MzEx9/fXXaty4sSTp3nvvVV5ent577z39+eefuuCCC4K2f7KpZGfTmR599FG1atVKJ0+e1L59+5wejl+4lNEPM2fOlMvl0q233mq7bocOHdSkSROtX79eSUlJqlSpki666CLNmTNHkrRixQq1bNlSYWFhatiwoRYvXuz1/IKu6V27dq2Sk5NVvXp1hYWFKTExUQMHDvR6Xl5env7xj3+oadOmqlixoqKjo9WlSxetXbv2nGM9cOCAHn30UTVt2lRVqlRReHi4UlJS9J///Cffuq+99poaN26sSpUq6YILLtCVV16pmTNneuoHDx7U0KFDVbduXbndbtWoUUP/93//p++//96zzpnXSi9fvlwul0s7duzQv//9b8/lMjt37jzntdKbNm1Sr169FBUVpYoVK+rKK6/Uv/71r3xj3bhxo6655hqFhYUpPj5ezz33XL6/OJ3L+vXr1b9/f9WrV08VK1ZUbGysBg4cqP3793utV5jXWxC3263Y2NhCjQU401dffaVOnTp5mjLpVKOflJSkBQsW6NChQ5bPJ5vIJl/58rsPpVd2drYkKSYmxmt5zZo1VaZMGVWoUMHy+WQT2VQYK1eu1Jw5czRhwoRCrR/qOGNWRLm5ufroo4/Upk2bfB/APJc///xT119/vfr27avevXtr8uTJ6tu3r2bMmKGhQ4dq0KBBuvXWWzV+/Hj16tVLaWlpqlq1aoHb2rt3rzp37qzo6Gg9+eSTioyM1M6dO/Xxxx97rXfXXXdp2rRpSklJ0d13360TJ07oq6++0qpVq3TllVcWuO3t27dr/vz56t27txITE5WRkaE33nhDSUlJ+umnnxQXFyfp1Kn0hx56SL169dLDDz+sY8eOaf369Vq9erXnF/agQYM0Z84cDRkyRJdccon279+vr7/+Wj///LOuuOKKfPu++OKLNX36dA0bNkzx8fF65JFHJEnR0dH6448/8q2/ceNGtW3bVrVq1dKTTz6pypUr66OPPlL37t01d+5c9ejRQ5KUnp6ujh076sSJE5713nzzTYWFhRXqe7do0SJt375dAwYMUGxsrDZu3Kg333xTGzdu1KpVq+RyuYr0egF/5eTkFPhzXKlSJR0/flwbNmxQq1atLLdBNpFNhVWU330onTp06KCXXnpJd911l0aPHq1q1arp22+/1eTJk/XQQw8V6rIzsolssnLy5Ek9+OCDuvvuu9W0adNCjSvkGRTJp59+aiSZ119/vVDrJyUlGUlm5syZnmWbNm0ykkyZMmXMqlWrPMu/+OILI8lMnTrVs2zq1KlGktmxY4cxxph58+YZSWbNmjXn3OfSpUuNJPPQQw/lq+Xl5Xn+XadOHdOvXz/P18eOHTMnT570Wn/Hjh3G7XabMWPGeJZ169bNNG7c2PJ1R0REmMGDB1uu069fP1OnTh2vZXXq1DFdu3bNN4azj8u1115rmjZtao4dO+b12tq0aWPq16/vWTZ06FAjyaxevdqzbO/evSYiIsLruJ7LkSNH8i374IMPjCSzcuVKz7LCvF47a9asyfc6gXNp2rSpadCggTlx4oRnWU5Ojqldu7aRZObMmWP5fLLp3Mim/Hz93YfS7dlnnzVhYWFGkufxl7/8pVDPJZvOjWw6ZeLEiSYiIsLs3bvXGHPqZ8bu+IY6LmUsopkzZ6p8+fLq06dPoZ9TpUoV9e3b1/N1w4YNFRkZqYsvvlgtW7b0LD/97+3bt59zW5GRkZKkBQsWKDc3t8B15s6dK5fL5fVB0NNO/6WiIG63W2XKnPrROHnypPbv368qVaqoYcOGXqeWIyMjtWvXLq1Zs8ZynKtXr9bu3bvPuU5RHThwQEuXLlWfPn108OBB7du3T/v27dP+/fuVnJysLVu2eO4Y9tlnn6lVq1Zq0aKF5/nR0dGF/ozEmX8hOnbsmPbt2+c5C3H2MQnW6wUK8sADD+iXX37RXXfdpZ9++kkbNmzQnXfeqT179kiSjh49arsNsimwSnI2FeV3H0qvunXrqn379nrzzTc1d+5cDRw4UC+88IImTpxYqOeTTYFVkrJp//79euaZZ/T0008rOjrap+eGMhqzIjh06JA++eQTJScne32uw058fHy+N3ZERIQSEhLyLZNOncI/l6SkJPXs2VOjR49W9erV1a1bN02dOlU5OTmedbZt26a4uDhFRUUVeozSqeurX3nlFdWvX19ut1vVq1dXdHS01q9fr6ysLM96TzzxhKpUqaIWLVqofv36Gjx4sL755huvbY0bN04bNmxQQkKCWrRooVGjRlkGpy+2bt0qY4znTXnm43So7t27V5L066+/qn79+vm2Udg71h04cEAPP/ywYmJiFBYWpujoaCUmJkqS1zEJ5usFCjJo0CA99dRTmjlzpho3bqymTZtq27ZtevzxxyWd+o+NHbKJbCqMov7uQ+k0a9Ys3XvvvXr77bd1zz336KabbtI777yjfv366Yknnsj3WaOCkE1k07n89a9/VVRUlB588MFCjae4oDErgvnz5xfpjlTnujvQuZYbY865LZfLpTlz5ig1NVVDhgzR77//roEDB6p58+a2H/a388ILL2j48OFq37693n//fX3xxRdatGiRGjdu7PWhz4svvlibN2/WrFmz1K5dO82dO1ft2rXz+ktTnz59tH37dr322muKi4vT+PHj1bhxY33++ed+jVGSZyyPPvqoFi1aVODjoosu8ns/0qnX8dZbb2nQoEH6+OOP9eWXX2rhwoVe4zi9XrBeL3Auzz//vDIyMvTVV19p/fr1WrNmjefnskGDBrbPJ5vIpsIo6u8+lE6vv/66Lr/8csXHx3stv/HGG3XkyBH98MMPttsgm8imgmzZskVvvvmmHnroIe3evdtzo5Njx44pNzdXO3fu1IEDBwLyOs47J6+jLK66dOliqlSpYg4fPlzo55zruteCrgk2xhhJXtfcnn2tdEFmzJhhJJm33nrLGGPM4MGDjcvlMvv377cc29nXSjdr1sx07Ngx33q1atUySUlJ59xOTk6O6dq1qylbtqw5evRogetkZGSYWrVqmbZt23qWFfVa6YyMDCPJjBgxwvL1GWNMgwYNTKtWrfItf+CBB2yP64EDB4wkM3r0aK/lv/zyi5FkRo4cec7nFvR67fAZMwTCVVddZeLj4/N97uFsZNMpZJO9ovzuQ+nVoEED07Jly3zLP/zwQyPJfP7555bPJ5tOIZvyW7ZsmdfnFgt6PPzww3YvMSRxxsxHf/zxhxYvXqwePXqoUqVKjo3jzz//zPeXocsuu0ySPKfle/bsKWOMRo8ene/5Zz/3TGXLls1Xnz17tue649POvgyhQoUKuuSSS2SMUW5urk6ePOl1ulqSatSoobi4OK9LB4qqRo0a6tChg9544w3P52nOdObdiK677jqtWrVK3333nVd9xowZtvs5/Ze5s4/J2bdmDfbrBQrrww8/1Jo1azR06FDP5x7OF7KpZGZTqPzuQ/HRoEED/fDDD/rll1+8ln/wwQcqU6aMLr300vM6HrKp5GRTkyZNNG/evHyPxo0bq3bt2po3b57uuusu23GGIm6X76MPP/xQJ06ccPxSjnfffVevv/66evTooQsvvFAHDx7UW2+9pfDwcF133XWSpI4dO+qOO+7Qq6++qi1btqhLly7Ky8vTV199pY4dO2rIkCEFbvv666/XmDFjNGDAALVp00b//e9/NWPGDNWrV89rvc6dOys2NlZt27ZVTEyMfv75Z02cOFFdu3ZV1apVlZmZqfj4ePXq1UvNmjVTlSpVtHjxYq1Zs0Z///vfA3IcJk2apHbt2qlp06a65557VK9ePWVkZCg1NVW7du3yzCHy+OOPa/r06erSpYsefvhhz21f69Spo/Xr11vuIzw8XO3bt9e4ceOUm5urWrVq6csvv9SOHTu81jt48KBfr3fixInKzMz0fAD2008/1a5duyRJDz74oOcaeuBMK1eu1JgxY9S5c2dVq1ZNq1at0tSpUz0/6+cb2XRKScomKXR+96H4eOyxx/T555/r6quv1pAhQ1StWjUtWLBAn3/+ue6++27PLeTPF7LplJKQTdWrV1f37t3zLT/d+BVUKzbO9ym64q5Vq1amRo0aXremLoxAn5L//vvvzS233GJq165t3G63qVGjhrn++uvN2rVrvbZz4sQJM378eNOoUSNToUIFEx0dbVJSUsy6deu8xnD2bV8feeQRU7NmTRMWFmbatm1rUlNTTVJSktcp+TfeeMO0b9/eVKtWzbjdbnPhhReaxx57zGRlZRljTp2if+yxx0yzZs1M1apVTeXKlU2zZs3y3WbZn9u+GmPMtm3bzJ133mliY2NN+fLlTa1atcz111+f7zbh69evN0lJSaZixYqmVq1a5tlnnzXvvPNOoW77umvXLtOjRw8TGRlpIiIiTO/evc3u3bu9TskX9vWeS506dc55St5ufCi9tm7dajp37myqV69u3G63adSokRk7dqzJyckp1PPJJrKpMIr6uw+l2+rVq01KSornPdCgQQPz/PPPm9zcXNvnkk1kk69Kwu3yXcZYnJsFAAAAAAQdnzEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADgs5CaYzsvL0+7du1W1alW5XC6nhwOgAMYYHTx4UHFxcSpTpnT8fYdsAkIf2UQ2AaGo0NkUrAnSJk6caOrUqWPcbrdp0aKFWb16daGel5aWds5Jdnnw4BFaj7S0tGBFSNCQTTx4lPxHccumouaSMWQTDx7F6WGXTUE5Y/bhhx9q+PDhmjJlilq2bKkJEyYoOTlZmzdvVo0aNSyfW7VqVUlSWlqawsPDgzE8AH7Kzs5WQkKC5/1aXJBNQMlWHLPJn1ySyCagOChsNrmMMSbQO2/ZsqWuuuoqTZw4UdKp0+wJCQl68MEH9eSTT1o+Nzs7WxEREcrKyiJggBBVXN+nZBNQshXH96k/uSQVz9cMlDaFfZ8G/ALs48ePa926derUqdP/dlKmjDp16qTU1NRA7w4ACoVsAhBqyCUAZwr4pYz79u3TyZMnFRMT47U8JiZGmzZtyrd+Tk6OcnJyPF9nZ2cHekgAQDYBCDm+5pJENgElmeO3LBo7dqwiIiI8j4SEBKeHBABkE4CQRDYBJVfAG7Pq1aurbNmyysjI8FqekZGh2NjYfOuPGDFCWVlZnkdaWlqghwQAZBOAkONrLklkE1CSBbwxq1Chgpo3b64lS5Z4luXl5WnJkiVq3bp1vvXdbrfCw8O9HgAQaGQTgFDjay5JZBNQkgXldvnDhw9Xv379dOWVV6pFixaaMGGCDh8+rAEDBgRjdwBQKGQTgFBDLgE4LSiN2c0336w//vhDzzzzjNLT03XZZZdp4cKF+T7cCgDnE9kEINSQSwBOC8o8Zv5gPg4g9JXG92lpfM1AcVMa36el8TUDxY1j85gBAAAAAHxDYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcFjAG7NRo0bJ5XJ5PRo1ahTo3QCAT8gmAKGGXAJwpnLB2Gjjxo21ePHi/+2kXFB2AwA+IZsAhBpyCcBpQXn3lytXTrGxscHYNAAUGdkEINSQSwBOC8pnzLZs2aK4uDjVq1dPt912m3777bdzrpuTk6Ps7GyvBwAEA9kEINT4kksS2QSUZAFvzFq2bKlp06Zp4cKFmjx5snbs2KGrr75aBw8eLHD9sWPHKiIiwvNISEgI9JAAgGwCEHJ8zSWJbAJKMpcxxgRzB5mZmapTp45efvll3XXXXfnqOTk5ysnJ8XydnZ2thIQEZWVlKTw8PJhDA1BE2dnZioiIKNbvU7IJKHmKezbZ5ZJENgHFUWGzKeifMI2MjFSDBg20devWAutut1tutzvYwwAAL2QTgFBjl0sS2QSUZEGfx+zQoUPatm2batasGexdAUChkU0AQg25BJRuAT9j9uijj+qGG25QnTp1tHv3bo0cOVJly5bVLbfcEuhdIUB+/vlny3r16tVtt/H+++9b1ufPn+/LkAq0adMmy/revXst6xdffLHtPlasWGFZj46Ott0GQhPZBCDUkEs4l+PHj1vWX331VdttPPfcc5b1a665xrL+8ccf2+6juPjjjz8s6yNHjrSsT5482XYfgfh0WMAbs127dumWW27R/v37FR0drXbt2mnVqlX8hxaAo8gmAKGGXAJwpoA3ZrNmzQr0JgHAb2QTgFBDLgE4U9A/YwYAAAAAsEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhAb8rI0KP3TxljzzyiGU9EPOY9ezZ07I+d+5c2324XC6/6ps3b7bdx7x58yzr9957r+02gOLuyJEjtuvYzdeyZ88e223YvZ/2799vWb/wwgtt95GQkGBZ79Wrl2X9qquust1HxYoVbdcBUHqkp6fbrvPjjz9a1seNG2dZX758uQ8jKtoYioulS5farjNw4EDL+m+//WZZt/s/ZqBwxgwAAAAAHEZjBgAAAAAOozEDAAAAAIfRmAEAAACAw2jMAAAAAMBhNGYAAAAA4DAaMwAAAABwGI0ZAAAAADiMCaZLgYsvvtiyfvfdd1vWN23aZLuPKVOmWNb37dtnWZ8zZ47tPuzYTXgbHR1tu42rr77a73EAoW748OGW9VmzZtlu48SJE5Z1u/d8IPz3v/+1XccuF1577TXLepcuXWz38c9//tOyHhsba7sNAMWH3YTGTz75pO021q5dG6jhFGt79+61Xef222+3rBdmsm2731mhgjNmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMOYxwy66aabgr6PlJQUy7rL5bLdht06PXr0sKy3b9/edh92c74BxUF6erplfeHChX49X7KfH6ww72k7SUlJlvWsrCzbbfzwww9+jcHuWElSixYtLOtz5861rF911VU+jQlA0e3fv992nVtuucWybjdvVnGZM8suu7Zv3267jXr16lnW58+fb1n/5JNPbPexePFi23WCrVOnTudlP5wxAwAAAACH0ZgBAAAAgMNozAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADvN5gumVK1dq/PjxWrdunfbs2aN58+ape/funroxRiNHjtRbb72lzMxMtW3bVpMnT1b9+vUDOW6EmHXr1lnWv/jiC8u63WS1hZGcnGxZv/fee/3eB0ITueQtKirKsn7o0KHzNBJrQ4YMsay//PLLlvXWrVv7PYaKFSta1v/xj3/YbuPvf/+7Zf3uu++2rC9ZssR2H9WrV7ddB6GHbDr/7CYj7t27t+02CjN5vdMuueQS23Ueeughy3og/l80c+ZMy3r//v0t6+drMu5KlSpZ1s98XxZkypQpARzNufl8xuzw4cNq1qyZJk2aVGB93LhxevXVVzVlyhStXr1alStXVnJyso4dO+b3YAGgIOQSgFBENgHwhc9nzFJSUpSSklJgzRijCRMm6K9//au6desmSXrvvfcUExOj+fPnq2/fvv6NFgAKQC4BCEVkEwBfBPQzZjt27FB6ero6derkWRYREaGWLVsqNTW1wOfk5OQoOzvb6wEAgVKUXJLIJgDBRTYBOFtAG7P09HRJUkxMjNfymJgYT+1sY8eOVUREhOeRkJAQyCEBKOWKkksS2QQguMgmAGdz/K6MI0aMUFZWlueRlpbm9JAAgGwCEJLIJqDkCmhjFhsbK0nKyMjwWp6RkeGpnc3tdis8PNzrAQCBUpRcksgmAMFFNgE4W0Abs8TERMXGxnrd+jc7O1urV68OyG2NAcBX5BKAUEQ2ATibz3dlPHTokLZu3er5eseOHfrxxx8VFRWl2rVra+jQoXruuedUv359JSYm6umnn1ZcXJzt/AAo3n7++WfLusvl8nsfdtu4+OKL/d4HiidyyVuFChUs66+88opl/cknn7Tdx5nHuyB16tSx3YbdHDqPPvqoZf3777+33Yedhg0bWtb79Oljuw27/JswYYJlffbs2bb7uP/++23XQeghmwLr999/t13H7j0bKnOUNW7c2LI+bNgwy3qvXr1s9+Hv2dT333/fdp1BgwZZ1s/HPGU9evSwXcdu/rpQuQuqz43Z2rVr1bFjR8/Xw4cPlyT169dP06ZN0+OPP67Dhw/r3nvvVWZmptq1a6eFCxfaTuIJAEVFLgEIRWQTAF/43Jh16NBBxphz1l0ul8aMGaMxY8b4NTAAKCxyCUAoIpsA+MLxuzICAAAAQGlHYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYT7flRGlzx9//GG7zh133GFZt5uDzOquVaetXbvWsn7FFVfYbgOA1LNnT8t6mzZtbLcRFxdnWd+5c6ftNpo2bWq7jr8aNWpkWV+0aJFlPSIiwnYfV111lWXdLt9effVV233cc889lvVy5fh1jpLv0KFDtutkZmYGfRx28zTefvvtttt46KGHLOvR0dE+jakgR44csazbzf+1cuVK233k5OT4NKazud1u23Xs7lr6yCOP2G6jTJnicS6qeIwSAAAAAEowGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACH0ZgBAAAAgMNozAAAAADAYcxICVvz5s2zXcduAmm7OoDQUbNmTdt13nnnHcv6kCFDbLdx7NixQo+pIC1btrRdZ+rUqZb16tWr+zUGSUpJSbGsV61a1bK+adMm23188803lvWkpCTbbQDFXWEmrg8PD7esZ2dn+72NxYsXW9YvvPBC233YscvHl156yXYbL7/8smXd7lgE4v9udpNYjxs3znYbgTiexQVnzAAAAADAYTRmAAAAAOAwGjMAAAAAcBiNGQAAAAA4jMYMAAAAABxGYwYAAAAADqMxAwAAAACHMY8Z/l979x9aZfn/cfy17euOs3RrmjsOt1wGrloZSNq0zFKcRqHpH0UQWpH9OJN0QWJoIf1xMvsdywhCLVJDcJn+oZg/NgJnuIwh5SiRtNwcFtts5Vzt+v7Rx4Mn133vPuc+u+7tPB9wwHPel/f99hq+2Hv3zrldtba2uq4xxiR1juLiYtc1w4YNS+ocAPzz+OOPO9anT5/ueoyGhgbH+pAhQxzr8+bNcz1HVlaW65pk5eXlWe8BSAcVFRWuax577DHH+jvvvON6DLd7iP3000+O9b7cd+vYsWOO9SeffNKxfvjwYddz9IelS5c61pcvX+5YHzdunI/dDHxcMQMAAAAAyxjMAAAAAMAyBjMAAAAAsIzBDAAAAAAsYzADAAAAAMsYzAAAAADAMgYzAAAAALDM833M6urqtG7dOjU0NKi5uVk1NTWaP39+rL548WJt2rQp7u9UVFRo9+7dSTcLO3bs2OG6JiMjI6V1wAm5FDw33HCDL2sGg2uvvdax3tbW1j+NoN+RTf3v/vvvd6z35T5mFy9edKw/8sgjjnW3+zxK0ieffOJY/+WXX1yPkazc3FzH+vbt212PceeddzrWs7OzPfWU7jxfMevs7NTEiRNVXV39n2vmzJmj5ubm2GPLli1JNQkATsglAEFENgHwwvMVs7lz52ru3LmOa0KhkMLhcMJNAYAX5BKAICKbAHiRkveYHTx4UKNHj9aECRP0zDPP6Ndff03FaQCgz8glAEFENgG4xPMVMzdz5szRggULVFJSohMnTujFF1/U3LlzdejQIWVlZV2xvqurS11dXbHnHR0dfrcEIM15zSWJbAKQemQTgMv5Ppg9/PDDsT/fcsstuvXWWzV+/HgdPHhQM2fOvGJ9NBrVmjVr/G4DAGK85pJENgFIPbIJwOVS/nH5119/vUaNGqUff/yx1/rKlSvV3t4ee5w+fTrVLQFIc265JJFNAPof2QSkN9+vmP3bzz//rF9//VVjxozptR4KhRQKhVLdBgDEuOWSRDYB6H9kE5DePA9mv//+e9xPck6ePKlvv/1W+fn5ys/P15o1a7Rw4UKFw2GdOHFCL7zwgm644QZVVFT42jgAXEIuAQgisgmAF54HsyNHjuiee+6JPa+qqpIkLVq0SOvXr1djY6M2bdqktrY2FRYWavbs2XrllVf46U6ANTQ0JFWXJGNMUj088cQTrmtKS0uTOgcGL3IJQTZ+/HjH+g8//OB6jG+++caxfvfdd3vqCf2DbOp///XevEsaGxtdj3Hvvfc61s+ePetYj0ajrucIgkWLFjnW3fYB/vM8mM2YMcPxm/A9e/Yk1RAAeEUuAQgisgmAFyn/8A8AAAAAgDMGMwAAAACwjMEMAAAAACxjMAMAAAAAyxjMAAAAAMAyBjMAAAAAsMzzx+Vj8KmpqXGsZ2RkJH0Ot2OsWrUq6XMAQBB1dXUlfYzTp0/70AmAsrIy1zVffPGFY33q1Kl+tZNSkUjEsf7OO+/0UyfoK66YAQAAAIBlDGYAAAAAYBmDGQAAAABYxmAGAAAAAJYxmAEAAACAZQxmAAAAAGAZgxkAAAAAWMZ9zKBz58451o0xrsdwW3Pttdd66gkABoujR4/abgGABx9++KHtFlwtXbrUdc26dev6oRP4iStmAAAAAGAZgxkAAAAAWMZgBgAAAACWMZgBAAAAgGUMZgAAAABgGYMZAAAAAFjGYAYAAAAAljGYAQAAAIBl3GAaqqmpcaxnZGQkfY4XX3wx6WMAQLo6fvy47RaAAaGtrc2xXlVV5XqMjRs3+tOMA7fvrdxuIP3GG2+4niMrK8tTT7CPK2YAAAAAYBmDGQAAAABYxmAGAAAAAJYxmAEAAACAZQxmAAAAAGAZgxkAAAAAWMZgBgAAAACWebqPWTQa1fbt23X8+HHl5ORo6tSpWrt2rSZMmBBbc+HCBT3//PPaunWrurq6VFFRoffff18FBQW+N4++qaurc6y3trY61vtyHzNjjGP9rrvucj0GkCiyCUAQkU3+cvt+RZImTZrkWP/ll1/8aicpV111lWP97bff7p9GECierpjV1tYqEomovr5ee/fuVXd3t2bPnq3Ozs7YmuXLl2vnzp3atm2bamtrdebMGS1YsMD3xgHgErIJQBCRTQC88HTFbPfu3XHPN27cqNGjR6uhoUHTp09Xe3u7PvroI23evFn33nuvJGnDhg268cYbVV9frzvuuMO/zgHgf8gmAEFENgHwIqn3mLW3t0uS8vPzJUkNDQ3q7u7WrFmzYmtKS0tVXFysQ4cOJXMqAOgzsglAEJFNAJx4umJ2uZ6eHi1btkzTpk1TWVmZJKmlpUXZ2dnKy8uLW1tQUKCWlpZej9PV1aWurq7Y846OjkRbAgCyCUAgkU0A3CR8xSwSiejYsWPaunVrUg1Eo1Hl5ubGHkVFRUkdD0B6I5sABBHZBMBNQoNZZWWldu3apQMHDmjs2LGx18PhsC5evKi2tra49WfPnlU4HO71WCtXrlR7e3vscfr06URaAgCyCUAgkU0A+sLTYGaMUWVlpWpqarR//36VlJTE1SdNmqQhQ4Zo3759sdeampp06tQplZeX93rMUCikESNGxD0AwAuyCUAQkU0AvPD0HrNIJKLNmzdrx44dGj58eOz3n3Nzc5WTk6Pc3Fw98cQTqqqqUn5+vkaMGKGlS5eqvLycTxayKBqNOtbd7lPWl/uYlZaWJlUHkkE2waZz58451v/666+kz7FixYqkj4H+Rzb566OPPnJdE5T7lAGJ8DSYrV+/XpI0Y8aMuNc3bNigxYsXS5LeeustZWZmauHChXE3SgSAVCGbAAQR2QTAC0+DmTHGdc3QoUNVXV2t6urqhJsCAC/IJgBBRDYB8CKp+5gBAAAAAJLHYAYAAAAAljGYAQAAAIBlDGYAAAAAYBmDGQAAAABYxmAGAAAAAJZ5+rh8DEytra2O9b58nK+bOXPmONaHDRuW9DkAoL/99ttvrmtuuukmx/rvv//uVztAWtuzZ4/tFoCU4ooZAAAAAFjGYAYAAAAAljGYAQAAAIBlDGYAAAAAYBmDGQAAAABYxmAGAAAAAJYxmAEAAACAZdzHLA1kZjrP3xkZGUnVAWCwOnXqlOuac+fOOdbdMtSPe0kC6eC2225zXVNXV5f6Rnzw0EMP2W4BAcQVMwAAAACwjMEMAAAAACxjMAMAAAAAyxjMAAAAAMAyBjMAAAAAsIzBDAAAAAAsYzADAAAAAMsYzAAAAADAMm4wnQZ6enoc637c3LS2tjbpYwBA0Bw/ftx2CwD+Z8GCBa5r3n333ZT3cc011zjW77rrLtdjvP766361g0GEK2YAAAAAYBmDGQAAAABYxmAGAAAAAJYxmAEAAACAZQxmAAAAAGAZgxkAAAAAWMZgBgAAAACWebqPWTQa1fbt23X8+HHl5ORo6tSpWrt2rSZMmBBbM2PGjCvuafXUU0/pgw8+8KdjeHbzzTc71h999FHHekZGhus5KioqPPUE+IlsQqoUFxen/BzZ2dmua0aOHJnyPuA/sslfbt/PSNJzzz2X9HlycnIc65WVlY71wsLCpHtAevJ0xay2tlaRSET19fXau3evuru7NXv2bHV2dsate/LJJ9Xc3Bx7vPbaa742DQCXI5sABBHZBMALT1fMdu/eHfd848aNGj16tBoaGjR9+vTY68OGDVM4HPanQwBwQTYBCCKyCYAXSb3HrL29XZKUn58f9/qnn36qUaNGqaysTCtXrtQff/zxn8fo6upSR0dH3AMAkkE2AQgisgmAE09XzC7X09OjZcuWadq0aSorK4u9/sgjj+i6665TYWGhGhsbtWLFCjU1NWn79u29HicajWrNmjWJtgEAccgmAEFENgFwk/BgFolEdOzYMX311Vdxry9ZsiT251tuuUVjxozRzJkzdeLECY0fP/6K46xcuVJVVVWx5x0dHSoqKkq0LQBpjmwCEERkEwA3CQ1mlZWV2rVrl+rq6jR27FjHtVOmTJEk/fjjj70GTCgUUigUSqQNAIhDNgEIIrIJQF94GsyMMVq6dKlqamp08OBBlZSUuP6db7/9VpI0ZsyYhBoEADdkE4AgIpsAeOFpMItEItq8ebN27Nih4cOHq6WlRZKUm5urnJwcnThxQps3b9Z9992nkSNHqrGxUcuXL9f06dN16623puQfAHcff/yx7RaAlCKbkCqXrl44Wb16tWN97dq1jnW3eyZJUmlpqesaBA/Z5K++3M/vrbfe6odOgNTwNJitX79e0j83Q7zchg0btHjxYmVnZ+vLL7/U22+/rc7OThUVFWnhwoVatWqVbw0DwL+RTQCCiGwC4IXnX2V0UlRUdMXd6wEg1cgmAEFENgHwIqn7mAEAAAAAksdgBgAAAACWMZgBAAAAgGUMZgAAAABgGYMZAAAAAFjGYAYAAAAAlnn6uHwAANJJVlaW65o1a9YkVQcAQOKKGQAAAABYx2AGAAAAAJYxmAEAAACAZQxmAAAAAGAZgxkAAAAAWMZgBgAAAACWBe7j8o0xkqSOjg7LnQD4L5f+f176/5oOyCYg+MgmAEHU12wK3GB2/vx5SVJRUZHlTgC4OX/+vHJzc2230S/IJmDgIJsABJFbNmWYgP1YqaenR2fOnNHw4cOVkZEh6Z8ps6ioSKdPn9aIESMsdziwsZf+Stf9NMbo/PnzKiwsVGZmevxG9L+zKV2/9qnCfvonnfeSbCKb/MZ++ied97Kv2RS4K2aZmZkaO3Zsr7URI0ak3RcyVdhLf6XjfqbLT6Mv+a9sSsevfSqxn/5J170km/6Rrl//VGE//ZOue9mXbEqPHycBAAAAQIAxmAEAAACAZQNiMAuFQnr55ZcVCoVstzLgsZf+Yj/TF197f7Gf/mEv0xtff3+xn/5hL90F7sM/AAAAACDdDIgrZgAAAAAwmDGYAQAAAIBlDGYAAAAAYBmDGQAAAABYFvjBrLq6WuPGjdPQoUM1ZcoUff3117ZbGhDq6ur0wAMPqLCwUBkZGfr888/j6sYYvfTSSxozZoxycnI0a9Ys/fDDD3aaDbhoNKrbb79dw4cP1+jRozV//nw1NTXFrblw4YIikYhGjhypq6++WgsXLtTZs2ctdYz+QDYlhmzyD9mE3pBN3pFL/iGXkhPoweyzzz5TVVWVXn75ZX3zzTeaOHGiKioq1Nraaru1wOvs7NTEiRNVXV3da/21117Tu+++qw8++ECHDx/WVVddpYqKCl24cKGfOw2+2tpaRSIR1dfXa+/everu7tbs2bPV2dkZW7N8+XLt3LlT27ZtU21trc6cOaMFCxZY7BqpRDYljmzyD9mEfyObEkMu+YdcSpIJsMmTJ5tIJBJ7/vfff5vCwkITjUYtdjXwSDI1NTWx5z09PSYcDpt169bFXmtrazOhUMhs2bLFQocDS2trq5FkamtrjTH/7N2QIUPMtm3bYmu+//57I8kcOnTIVptIIbLJH2STv8gmkE3JI5f8RS55E9grZhcvXlRDQ4NmzZoVey0zM1OzZs3SoUOHLHY28J08eVItLS1xe5ubm6spU6awt33Q3t4uScrPz5ckNTQ0qLu7O24/S0tLVVxczH4OQmRT6pBNySGb0hvZlBrkUnLIJW8CO5idO3dOf//9twoKCuJeLygoUEtLi6WuBodL+8feetfT06Nly5Zp2rRpKisrk/TPfmZnZysvLy9uLfs5OJFNqUM2JY5sAtmUGuRS4sgl7/7PdgPAQBKJRHTs2DF99dVXtlsBgBiyCUDQkEveBfaK2ahRo5SVlXXFp7ScPXtW4XDYUleDw6X9Y2+9qays1K5du3TgwAGNHTs29no4HNbFixfV1tYWt579HJzIptQhmxJDNkEim1KFXEoMuZSYwA5m2dnZmjRpkvbt2xd7raenR/v27VN5ebnFzga+kpIShcPhuL3t6OjQ4cOH2dteGGNUWVmpmpoa7d+/XyUlJXH1SZMmaciQIXH72dTUpFOnTrGfgxDZlDpkkzdkEy5HNqUGueQNuZQkyx8+4mjr1q0mFAqZjRs3mu+++84sWbLE5OXlmZaWFtutBd758+fN0aNHzdGjR40k8+abb5qjR4+an376yRhjzKuvvmry8vLMjh07TGNjo5k3b54pKSkxf/75p+XOg+eZZ54xubm55uDBg6a5uTn2+OOPP2Jrnn76aVNcXGz2799vjhw5YsrLy015ebnFrpFKZFPiyCb/kE34N7IpMeSSf8il5AR6MDPGmPfee88UFxeb7OxsM3nyZFNfX2+7pQHhwIEDRtIVj0WLFhlj/vn419WrV5uCggITCoXMzJkzTVNTk92mA6q3fZRkNmzYEFvz559/mmeffdZcc801ZtiwYebBBx80zc3N9ppGypFNiSGb/EM2oTdkk3fkkn/IpeRkGGNMaq/JAQAAAACcBPY9ZgAAAACQLhjMAAAAAMAyBjMAAAAAsIzBDAAAAAAsYzADAAAAAMsYzAAAAADAMgYzAAAAALCMwQwAAAAALGMwAwAAAADLGMwAAAAAwDIGMwAAAACwjMEMAAAAACz7f0BbeTuDvrwZAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "misclassified = eval_data.get_misclassified_triples_pred_true_input()\n", "fig, axs = plt.subplots(nrows=3, ncols=3, figsize=(9,9))\n", "for i, (predClass, trueClass, input) in enumerate(misclassified[:9]):\n", " axs[i//3][i%3].imshow(reshape_2d_image(input), cmap=\"binary\")\n", " axs[i//3][i%3].set_title(f\"{trueClass} misclassified as {predClass}\")\n", "plt.tight_layout()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "While some of these examples are indeed ambiguous, there still is room for improvement." ] } ], "metadata": { "interpreter": { "hash": "9b3442ae4bdb9561e722e28424c33a03c16d40b3aa50369b79d367cad7b1adea" }, "kernelspec": { "display_name": "Python 3.8.13 ('sensai')", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.18" } }, "nbformat": 4, "nbformat_minor": 2 }