{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Autodiff And Linear Regression" ], "id": "site-title" }, { "cell_type": "markdown", "id": "c4113faf", "metadata": {}, "source": [ "## Autodiff with a Linear Regression example\n", "\n", "So far, we've seen all the fine Jax transformations. Now let's see how we can apply them. In this example, I'm going to use a very simple linear regression model and estimate the parameters using gradient descent.\n", "\n", "But first, let's complete the ritual of creating keys for the PRNG. (Seems kinda annoying that you've to do this every time but that's how Jax is, sadly.)" ] }, { "cell_type": "code", "execution_count": 1, "id": "ea76df87", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2026-05-07 19:27:21.548\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36msetup_backend\u001b[0m:\u001b[36m24\u001b[0m - \u001b[1mSetting up JAX to use the `mps` backend.\u001b[0m\n", "\u001b[32m2026-05-07 19:27:21.548\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36msetup_backend\u001b[0m:\u001b[36m26\u001b[0m - \u001b[32m\u001b[1mJAX Platform set to use the `mps` backend.\u001b[0m\n", "Platform 'mps' is experimental and not all JAX functionality may be correctly supported!\n", "\u001b[32m2026-05-07 19:27:21.716\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36mlog_device\u001b[0m:\u001b[36m14\u001b[0m - \u001b[1mXLA DEVICE: mps\u001b[0m\n" ] } ], "source": [ "# setting up device backend for apple silicon (mps)\n", "# if you have it that is.\n", "\n", "from utils import setup_backend, log_device\n", "\n", "setup_backend()\n", "log_device()" ] }, { "cell_type": "code", "execution_count": 2, "id": "c52f3329", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:57.859139Z", "start_time": "2023-02-12T22:45:57.086833Z" } }, "outputs": [], "source": [ "import jax\n", "import jax.numpy as jnp\n", "\n", "masterkey = jax.random.key(0)\n", "masterkey, *subkeys = jax.random.split(masterkey, 10)" ] }, { "cell_type": "markdown", "id": "e4daa74a", "metadata": {}, "source": [ "## Function to generate data\n", "\n", "The function basically is:\n", "\n", "$$\n", "f(x) = 4x + 3\n", "$$" ] }, { "cell_type": "code", "execution_count": 3, "id": "9e2e9aba", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:57.865224Z", "start_time": "2023-02-12T22:45:57.860200Z" } }, "outputs": [ { "data": { "text/plain": [ "7" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "f = lambda x: 4 * x + 3\n", "f(1)" ] }, { "cell_type": "markdown", "id": "1cd9767d", "metadata": {}, "source": [ "The function is defined, now to get some inputs for the function to generate the outputs." ] }, { "cell_type": "code", "execution_count": 4, "id": "18fab3ce", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.035339Z", "start_time": "2023-02-12T22:45:57.866003Z" } }, "outputs": [], "source": [ "xs = jax.random.normal(subkeys[0], shape=(1000, ))\n", "ys = f(xs)" ] }, { "cell_type": "markdown", "id": "dd6508c0", "metadata": {}, "source": [ "If you've your doubts over f being linear, well, there's always matplotlib!" ] }, { "cell_type": "code", "execution_count": 5, "id": "228dbd7c", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.553576Z", "start_time": "2023-02-12T22:45:58.036403Z" } }, "outputs": [ { "data": { "text/plain": [ "[]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGdCAYAAAA8F1jjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAL71JREFUeJzt3Ql4TNf/x/FvFgklsddarVJUa68ltqRo0db2a6uqi6JFqW5iiS1iC0kp1VYtbWxdbLXUFnttsasqftWiShDUktQWYv7Pvf+fqVNCwsycOzPv1/Pch/NN8M2ZSebjnHvv+IiITQAAACzIV3cDAAAA6SGoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAy/IXD1C4cGFJSUnR3QYAAMiEoKAgOXr0qGcHFSOkJCYm6m4DAADchSJFitw2rLh9ULm+kmJ8oayqAADgPqspxkLDnV673T6oXGd8oQQVAAA8CyfTAgAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAACAW3q0bi15st2rkjUoh+jiMe+eDAAAHOfdaRPkwQqPm79PPvmXbPthsehAUAEAAHbB+fNJ5Mof/imIyC8rfhRd2PoBAACmqs2eUULK+bPnJLxCLbl84YLowooKAACQbrOnSuFSJe0z8cOIT2X1pK+1zwxBBQAAL5arYAHpt2yuUot+rqWcOnRYrICtHwAAvFSNF5srIeVs0gkJL1/TMiHFwIoKAABexsfHR3otmCH5ihW11+ZEj5R138wUqyGoAADgRfIULSx9Fs9WakMaPy+njxwVK2LrBwAAL1G79YtKSDn5x5/mVo9VQ4qBFRUAALxgq6ffsnmSs0B+e21m1DDZOGueWB1BBQAAD5bvwQckYsEMpTaoQTPzxFl3wNYPAAAe6sm2ryghJfG/+6RbuRC3CSkGVlQAAPAwPr6+MnDtYrkvONhe+7bPINk6f5G4G4IKAAAepECJ4tJj7jdKLapeE0k+eUrcEVs/AAB4iKc6tVNCyh8/7TK3etw1pBhYUQEAwM35+vtJ9OZV4p8li702tXs/+WnJcnF3BBUAANxYoVIlJXz2VKUWGfqM/H36jHgCtn4AAHBTjd/tpISU3zZuNbd6PCWkGFhRAQDAzfj5+0vMjrVKbdL7vWTXih/F0xBUAABwI0XLlpEPpscptX61G8qFc8niidj6AQDATTTt8Z4SUvb8uN7c6vHUkGJgRQUAAIvzDwiQ4dvUbZ2JnbvJ3rUbxNMRVAAAsLCHKpSTrtPGK7U+NZ+SSyl/izdg6wcAAIt6vm93JaTsXLrS3OrxlpBiYEUFAACLCciW1byB243GdXhX9iVsEW9DUAEAwEIefqKSdIn7XKn1rl5fLl+4IN6IoAIAgEW0GtxXqjZ71j7eMm+RfNd3kHgzp56jUqdOHZk/f74kJiaKzWaTZs2aKR+Pi4sz6zceixcvdmZLAABYTuB998mIXQlKSPm8bWevDylODyrZs2eXnTt3SpcuXdL9HCOYFCxY0H68/PLLzmwJAABLKRVSVYZuWqHUIqo9Kfu37tDWk9ds/SxZssQ8bufy5cuSlJTkzDYAALCk12IHScVGDezjDTPmyOxBMVp7shrt56iEhYWZQeXMmTOycuVK6du3r5w+fTrdzw8ICJDAwED7OCgoyEWdAgDgGFmDcsiQDcuU2phXO8gfO3cxxVa6j4qx2vL6669L/fr1pWfPnhIaGmpuBfn6pt9WRESEJCcn2w/j/BcAANzFo3Vq3hRSej4RRkhJh4+I2MQFjBNlmzdvLvPmzUv3c4oXLy4HDhwwg4uxupLRFRUjrAQHB0tKSopTegcAwBHajYmVx8Jq28drpk6XeTGjvHJyg4KCzAWHO71+a9/6udHBgwfl5MmTUrJkyXSDSmpqqnkAAOAu7ssZLIPWxSu1Ua3ayeHde7X15C4sFVSKFCkiefPmlWPHjuluBQAAh3i8Xqi0HT1MqfWoXFfSrlxhhnUHFePyZGN15MatnQoVKpgnyxpHZGSkzJ49W44fPy4lSpSQmJgY+f333yU+Xk2dAAC4o04TxsgjNZ6wj1dMnCKLRo/V2pM7sjnrCA0Ntd1KXFycLWvWrLYlS5bYkpKSbJcvX7YdPHjQNm7cONv999+fqX8jKCjI/DuNX535tXAwBzwHeA7wHOA5kNHnQI48uW0jdiUoR+HSj/Acksy/frvsZFrdJ+MAAOAKxn1RjPujXHc1NVUiqteTa1fTeADc/WRaAADcWdep4+WhiuXs4/jPJsjSL77S2pO7I6gAAHCPgvLllQGrFii1mOatJWn/QebWnW/4BgCAu6vSpLESUi4mp0j3irUJKQ7CigoAAHep26wpUrj0I/bxgo8/k1VfTWM+HYigAgBAJuUqcL/0W67eaT36uZZy6tBh5tLB2PoBACATqj/fVAkp506clPAKtQgpTsKKCgAAGdTrh+mS/6Fi9vHcYR/L2q9nMH9ORFABAOAO8hQpJH2WfK/UhjR+Xk4fOcrcORlbPwAA3Ebt1i8oIeXkocMSXr4mIcVFWFEBAOAWfHx8pO/SOZKrYAF7bdbAGEmYOYf5ciGCCgAA/5KvWFGJWDhTqQ16qrmcPZ7EXLkYWz8AANwgrE1rJaQc3fe7dCsXQkjRhBUVAACMrR5fXxm4ZrHclzPYPh/f9R0kW+YtYn40IqgAALxegYcfkh7zvlXmIapeE0k+ecrr50Y3tn4AAF6tQYc3lJByaOcv5lYPIcUaWFEBAHglX38/GbpxhWQJDLTXpvXoLzsWL9PaF1QEFQCA1ylUqoSEz1bfPDAy7Bn5+68z2nrCrbH1AwDwKo27dlRCyu+bt5lbPYQUa2JFBQDgFfz8/SVmx1qlNumDCNm1fLW2nnBnBBUAgMcrWraMfDA9Tqn1q91QLpxL1tYTMoatHwCAR2sS3lUJKXvWrDe3eggp7oEVFQCAR/IPCJDh235UahO7hMveNeu19YTMI6gAADxOsfKPyXtfT1RqfWs9LReTU7T1hLvD1g8AwKP8p0+4ElJ2Ll1pbvUQUtwTKyoAAI+QJWugDNuiXsEzrsN7si9hs7aecO8IKgAAt/dwlYrSZdJYpda7Rn25fP6Ctp7gGAQVAIBbe2lQH6nW/Dn7eOsPi+Xb3gO19gTHIagAANxSQLZsEr15pVL7vF0X2b9lu7ae4HgEFQCA23mkRlXpNOETpRZR7UlJvXhJW09wDoIKAMCtvBozUCo1fso+Tpg1V2ZFDdfaE5yHoAIAcAtZc2SXIQnLldqY1zrKHz/9rK0nOB9BBQBgeWXqhMhbn49Uaj2fCJOrly9r6wmuQVABAFha20+Gy+NP1rWP10ybLvOGj9LaE1yHoAIAsKRswcEyeH28Uhv1cns5/MsebT3B9QgqAADLebxeXWk7Wj1BtkflupJ25Yq2nqAHQQUAYCkdx4+WUiHV7OOVX06RhaPUu87CexBUAACWkD13Lhm4ZrFSG/liG0n87z5tPUE/ggoAQLsKDevL6x8Nto/TrlyVXtXC5NrVNK19QT+CCgBAq3emjJPilcrbx/GfT5SlY7/U2hOsg6ACANAiKF9eGbBqgVKLbfGKHP/9AI8I7Hz/+S0AAK5R5blGSki59Pd56V6xNiEFN2FFBQDgUh9MnyRFy5a2jxeO+lxWfjmVRwG3RFABALhEzgL5pf/y+UptWJOX5OQff/IIIF1s/QAAnK76802VkJJ88pSEV6hFSMEdsaICAHCqnvO/k/uLP2gfz4sZLWumfsesI0MIKgAAp8hduKD0jZ+j1IY2fkH+OpLIjCPD2PoBADhcrZdfUELKqcNHJLx8TUIKMo0VFQCAw/j4+EjfpXMkV8EC9tqsQTGSMENdWQEyiqACAHCIvA8Uld6LZiq1wU+3kDPHjjPDuGts/QAA7lno6y8rIeXYb/ulW7kQQgruGSsqAIC75uPrK1GrF5rvfHzd9H5DZPNc9db4wN0iqAAA7opxybFx6fGNouo3leQTJ5lROAxbPwCATKv/VhslpPy5a4+51UNIgVsFlTp16sj8+fMlMTFRbDabNGvW7KbPiYqKkqNHj8qFCxdk2bJlUrJkSWe2BAC4B75+fhK9eZU8824ne+3rXpEyunV75hXuF1SyZ88uO3fulC5dutzy4z169JB3331XOnXqJNWrV5fz589LfHy8BAYGOrMtAMBdKFSqhMT+tE4CsmW11yLDnpHtC5cyn3AqmysOQ7NmzZTa0aNHbd26dbOPg4ODbRcvXrS99NJLGf57g4KCzL/b+NVVXwsHc8BzgOeAtz0HGr3TwTZiV4L9ePvLT7X3xCFuPQcZff3WdjJt8eLFpVChQrJ8+XJ7LTk5WTZt2iQhISEyffr0W/65gIAAZcUlKCjIJf0CgDfy8/eXmB1rldrkD3vLz8tWaesJ3kXbybQFCxY0f01KSlLqxvj6x24lIiLCDDTXD+P8FwCA4xUtW/qmkNK/TiNCClzK7a76iY6OluDgYPtRpEgR3S0BgMd57sN35IPpk+zjvesSzKt6zp89p7UveB9tWz/Hj///LZULFChg//318U8//ZTun0tNTTUPAIDj+QcEyPBtPyq1r7p2l92r1zHd8K4VlYMHD8qxY8ekfv36yvkmxtU/CQkJutoCAK9VrPxjN4WUvrWeJqTAc1dUjMuTb7wvinECbYUKFeT06dNy+PBhGTVqlPTt21d+++03M7gMGjTIvKfK3LlzndkWAOBfWvTuJrVffsE+Nk6WNU6aBazAaZcehYaG2m4lLi7O/jlRUVG2Y8eOmZclL1u2zPbII4845fImDuaA5wDPAZ4DNz8HsmQNVC47No7SNaszV3y/2KxyebLP/37jtoztIuPqH+PE2pSUFN3tAIDbKF65grwz+Qul1rtGfbl8/oK2nuA9gjL4+u12V/0AAO5dy6jeSkjZtmCJeVUPIQVWw7snA4AXCciWTaI3r1RqY9u/I79v3qatJ+B2CCoA4CUeqf6EdJo4RqlFVKsnqRcvausJuBOCCgB4gVeGR0nlZ562jzfOmiczo4Zp7QnICIIKAHiwrDmyy5CEf95TzfDp6x3l4I6ftfUEZAZBBQA8VJnaNeStsR8rtZ5PhMnVy5e19QRkFkEFADxQ29HD5PF6ofbx2q9nyNxhamgB3AFBBQA8SLbgYBm8Pl6pjXq5vRz+ZY+2noB7QVABAA/x2JN1pN0nMUqtR+W6knbliraegHtFUAEAD9Dhi4+ldK0a9vGqr6bJgo8/09oT4AgEFQBwY9lz55KBaxYrtZEt20ji3n3aegIciaACAG6q/NP1pM2IIfbxtWvXpFeVUEm7elVrX4AjEVQAwA11mTRWHq5S0T5e+sVXEv/ZBK09Ac5AUAEANxKUN48MWL1QqcW2eEWO/35AW0+AM/HuyQDgJio/11AJKZfOn5fuFWsTUuDRWFEBADfw/vQ4eaBsGft44aixsvLLKVp7AlyBoAIAFhZ8f36JXDFfqQ1v2kpOHDykrSfAldj6AQCLqtaiiRJSUv46LeEVahFS4FVYUQEAC+ox71sp8PBD9vG82NGyZsp3WnsCdCCoAICF5C5UUPounaPUhj7zovx1+Ii2ngCd2PoBAIuo1ep5JaScTjwm4eVrElLg1VhRAQDNfHx8pPeS2ZKncCF7bfbgWNkw/XutfQFWQFABAI3yPlBUei+aqdQGP91Czhw7rq0nwErY+gEATeq+3koJKcbdZbuVCyGkADdgRQUAXMzH11cGrFogOfLkttem9x8qm+f8wGMB/AtBBQBc6P7iD0rP+eplxlH1m0ryiZM8DsAtsPUDAC5S/802Skj585c95lYPIQVIHysqAOBkvn5+MnjDMgm8L5u99nXEANm+IJ65B+6AoAIATlTwkRLS/ftpSm1A2LPm7fAB3BlbPwDgJA27vKWElP3bdphbPYQUIONYUQEAB/Pz95eYHWuV2uRufeTnpSuZayCTCCoA4EBFHi0lH86YrNT6120s58+cZZ6Bu8DWDwA4yHMfdFFCyq/rN5pbPYQU4O6xogIA98gvSxaJ2b5GqX31bg/ZvUrd/gGQeQQVALgHxcqVlfe++VKp9a3VUC4mJzOvgAOw9QMAd6lFxIdKSNm14kdzq4eQAjgOKyoAkElZsgbKsC2rldr4Th+Y56QAcCyCCgBkQvFK5eWdKeOUWp+QBnLp7/PMI+AEbP0AQAa9GNlLCSnbF8abWz2EFMB5WFEBgDsIyJZNojerN2v74s2u8tumrcwd4GQEFQC4jZLVqsjbX36q1CKq1ZPUixeZN8AFCCoAkI7W0ZFS5blG9vGm73+QGZFDmS/AhQgqAPAvgdnvk6EbVyi1T9t0koPbdzJXgIsRVADgBqVr1ZAOX3yszEmvqmFy5dJl5gnQgKACAP/T5uNoKd8gzD4f676ZKXOiRzI/gEYEFQBeL1twkAxev1SZh9Gt28ufu/Z4/dwAuhFUAHi1x8JqS7sxsUqtZ5VQuZqaqq0nAP8gqADwWm+OHSmP1g6xj1fFfS0LRqqXIgPQi6ACwOtkz5VTBq5dotRGtmwjiXv3aesJwK0RVAB4lfJP15M2I4YotR6V6kja1avaegKQPoIKAK/ROe5zKfFEJft42bg4WfLpeK09Abg9ggoAj5cjb26JWr1IqcX+51U5/tt+bT0ByBjePRmAR6v87NNKSLl84aJ0r1ibkAK4Ce1BJTIyUmw2m3Ls3btXd1sAPMB7334prwyLso8XffKF9K5eT66lpWntC4Cbbf388ssv0qBBA/v4Kie1AbgHwffnl8gV85Xa8Kat5MTBQ8wr4GYsEVSMYJKUlKS7DQAeoFrz5+SlQX3s479Pn5EBTz4ntmvXtPYFwI2DyiOPPCKJiYly6dIlSUhIkIiICDl8+PAtPzcgIEACAwPt46CgIBd2CsDKus/5WgqWfNg+nh/7ifw45VutPQFw83NUNm3aJG+88YY0atRI3n77bSlevLisXbtWcuTIccvPN0JMcnKy/TACDgDvlrtQQRmxK0EJKUOfeZGQAngAHxGxiYXkzJlTDh06JB9++KF89dVXGVpRMcJKcHCwpKSkuLhbALrVfOk/8nzf7vbx6aPHZGij580T8wFYl/H6bSw43On12xJbPzc6d+6c7Nu3T0qWLHnLj6emppoHAO/m4+MjEYtmSd6ihe2174d8JOu/m621LwAetvXzb9mzZ5cSJUrIsWPHdLcCwKLyFi0iH/28QQkpgxu2IKQAHkh7UImNjZW6devKgw8+KCEhITJnzhxJS0uTb7/lBDgAN6v7WivpvXiWfZx04A/pVi5Ezhw9znQBHkj71k/RokXNUJI3b145efKkrFu3TmrUqCGnTp3S3RoAC/Hx9ZXIlT9IUN489tqMyKGy6fsftPYFwMODyssvv6y7BQAWl/+hYtLrh+lKbWCDpnIu6aS2ngB4ydYPANxOvfavKSHl8J7/mls9hBTAO2hfUQGAW/H185NB6+Mla/bs9to3EVGybcESJgzwIgQVAJZj3LjNuMvsjQaEPSspf53W1hMAPdj6AWApDTu/qYSUA9t+Mrd6CCmAd2JFBYAl+Pr7yfCtP5pbPtdNCe8rO+NXaO0LgF4EFQDaFSlTSj6cOVmp9a/bWM6fOautJwDWwNYPAK2e/aCzElJ+3bDJ3OohpAAwsKICQAu/LFkkZvsapRb3Xk/5ZaVaA+DdCCoAXO6Bx8vK+99+qdT61mooF5OTeTQAKNj6AeBSzXq+r4SUX1atMbd6CCkAboUVFQAu4R8YKMO3rlZqEzp/KP9dm8AjACBdBBUATvdQxfLSdeo4pdYnpIFc+vs8sw/gttj6AeBUL0T2VELKjkVLza0eQgqAjGBFBYBTBGTLKtGbVym1L956V37buIUZB5BhBBUADleiamXp/NVnSi2iWj1JvXiR2QaQKQQVAA718tD+8kSTxvbx5jkLZHr/IcwygLtCUAHgEIHZ75OhG9X35fnsjbfNNxUEgLtFUAFwz0rXrC4dxo1Sar2qhsmVS5eZXQD3hKAC4J68PmKIVHi6nn28/rvZ8v2Qj5hVAA5BUAFwV7IFB8ng9UuV2uhX3pQ/f97NjAJwGIIKgEwrG1pb2n8aq9R6VgmVq6mpzCYAhyKoAMiU9p99JGXr1rKPV0/6Rn4YMYZZBOAUBBUAGZI9V04ZuHaJUvv4pTfkyJ5fmUEATkNQAXBH5RqEyRsfRyu1HpXqSNrVq8weAKciqAC4rbe/+kxKVq1sHy8bHydLxoxn1gC4BEEFwC3lyJtbolYvUmofPf+qHNu3nxkD4DK8ezKAm1R65mklpBg3buteqTYhBYDLsaICQPHeN19KsXJl7ePFY8bJ8vGTmCUAWhBUAJiC8+eTyJU/KLMxvGkrOXHwEDMEQBu2fgBI1ebPKiHl/NlzEl6hFiEFgHasqABeLvz7aVLokRL28Q8fjZHVk7/R2hMAXEdQAbxUroIFpN+yuUot+tkX5dSfR7T1BAD/xtYP4IVCXmyhhJQzx45LePmahBQAlsOKCuBFfHx8pNfCGZLvgaL22pzoEbLum1la+wKA9BBUAC+Rp2hh6bN4tlIb0ug/cjrxmLaeAOBO2PoBvECdV19SQopxyXG3ciGEFACWx4oK4MF8fH2l/4r5Epwvr702Y0C0bJo9X2tfAJBRBBXAQz1Y4XF5d9oEpTaoQTM5m3RCW08AkFkEFcADtRsTK4+F1baPE/fuk5Et22jtCQDuBkEF8CC+/n4Su2OdUpsXM1rWTP1OW08AcC8IKoCHKPFEJekc97lSi36upZw6dFhbTwBwrwgqgAfoNHGMPFL9CaVmXNUDAO6OoAK4Mb8sWSRm+xqltmxcnCz5dLy2ngDAkQgqgJsqXauGdPjiY6U2uGELOXP0uLaeAMDRCCqAG3rvmy+lWLmySo2tHgCeiKACuJEsWQNl2JbVSm3hqLGy8ssp2noCAGciqABu4rEn60i7T2KUWlT9ppJ84qS2ngDA2QgqgBvoMe9bKfDwQ0qNrR4A3oCgAlhY4H33ydBNK5Ta3OGjZO206dp6AgBXIqgAFlWxYX157aPBSi0y7Bn5+68z2noCAFcjqAAW1G/5PMlV4H77+FpamnSv+M979wCAtyCoABaSNSiHDNmwTKnNHDhcNs6cq60nANCJoAJYRNVmz0irwf2UWr/aDeXCuWRtPQGAbr5iAZ07d5aDBw/KxYsXZePGjVK1alXdLQEuNXjDMiWkXEhONq/qIaQA8Hbag0rLli1l5MiREhUVJZUrV5adO3dKfHy85M+fX3drgNNlz5VTRuxKkGxBOey1b3oPlH61GjL7ACAiPiJi0zkTxgrKli1bpGvXrubYx8dHDh8+LGPGjJHhw4ff8c8HBQVJcnKyBAcHS0pKigs6Bhwj5MUW8kL/HkqtT82n5FLK30wxAI8XlMHXb63nqGTJkkWqVKki0dHR9prNZpPly5dLSAhvUQ/PZayi3OjMseMy+OkW2voBAKvSGlTy5csn/v7+kpSUpNSNcZkyZW75ZwICAiQwMFBJZIC7CMqbRwasXqjUpoT3lZ3x6k3dAAAWOUclsyIiIsyloutHYmKi7paADKn7WqubQkpEtXqEFACw6orKqVOn5OrVq1KgQAGlboyPHz9+yz9jbBMZJ9/euKJCWIG7bfUc//2AxLZ4RVs/AOAutK6oXLlyRbZt2yb169e314yTaY1xQoL6g/261NRU86SbGw/AqnIWyH9TSPmqa3dCCgC4yw3fjNWRyZMny9atW2Xz5s3y/vvvS/bs2SUuLk53a8A9qf9WG3nm3U5KrVfVMLly6TIzCwDuElRmzJhh3jNl4MCBUrBgQfnpp5+kUaNGcuLECd2tAXft36soh37eLZ+88iYzCgDudh+Ve8V9VGAleYoUkj5Lvldq4zq8J/sSNmvrCQCsyC3uowJ4ksZdO0qDDm8otR6V60ralSvaegIAd0dQAZyw1bNv4xYZ99a7zC0A3COCCnAP8j9UTHr9MF2pfda2sxzYuoN5BQAHIKgAd6lpj/ck9LVWSq17pdpy7WoacwoADkJQARyw1fPLqjUS925P5hIAHIygAmRCwZIPS/c5Xyu10a+8KX/+vJt5BAAnIKgAGfRCZE8JeaG5UguvUEts164xhwDgJAQV4A6Mt3X46OcNSm37wnj5utcA5g4AnIygAtxGkUdLyYczJiu1kS3bSOLefcwbALgAQQVIxyvDBkjlZxsqtfDyNcVmc+ubOQOAWyGoAP/i4+srH+1cr9QSZs6VWQOHM1cA4GIEFeAGxco/Ju99PVGZk9gWr8jx3w8wTwCgAUEF+J+2o4fJ4/VClfnoVi6E+QEAjQgq8Hq+/n4Su2OdMg8/Tv1O5seM9vq5AQDdCCrwag8/UUm6xH2u1KKfaymnDh3W1hMA4B8EFXitjhM+kVI1qio1tnoAwFoIKvA6flmySMz2NUpt+fhJsnjMOG09AQBujaACr1IqpJp0HK+eezKk0X/kdOIxbT0BANJHUIHXeHfaBHmwwuNKja0eALA2ggo8XpasgTJsy2qltuiTL2TFBPXW+AAA6yGowKM9FlZb2o2JVWoDGzSVc0kntfUEAMg4ggo8Vvc5X0vBkg8rNbZ6AMC9EFTgcQKyZZPozSuV2ryY0bJm6nfaegIA3B2CCjxKhYb15fWPBiu1AWHPSspfp7X1BAC4ewQVeIy+S+dI7kIFlRpbPQDg3ggqcHtZg3LIkA3LlNqsgTGSMHOOtp4AAI5BUIFbq9KksbQe2l+p9a/TSM6fPaetJwCA4xBU4LYGr18q2YKD7OOLKX9L35pPae0JAOBYBBW4nftyBsugdfFK7ds+g2Tr/EXaegIAOAdBBW6lxovN5cX+PZVan5pPyaWUv7X1BABwHoIK3EbszvXi6+trH59NOiGDGjTT2hMAwLkIKrC8HHlzS9RqdVtnanhf+Sl+hbaeAACuQVCBpdV59SVp3vN9pda7en25fOGCtp4AAK5DUIFljdiVoIyTDvwhMc1e1tYPAMD1CCqwnJwF8kv/5fOV2lfv9pDdq9Zq6wkAoAdBBZZSr/3r8uz7byu1XlXD5Mqly9p6AgDoQ1CBZbd6/ty1R0a3bq+tHwCAfgQVaJe7cEHpG6++L8/4ju/Lrxs2aesJAGANBBVo1eidDvJUx7ZKrUflupJ25Yq2ngAA1kFQgWW2en7btFW+eLOrtn4AANZDUIHL5StWVCIWzlRqn7ftLPu37uDRAAAoCCpwqSbhXSWsTWul1r1Sbbl2NY1HAgBwE4IKtG31GPdFMe6PAgBAeggqcLoCJYpLj7nfKLVPXn1LDu38hdkHANwWQQVO9Xy/HlKzZQul1r1ibbmWxlYPAODOCCpwCh8fH/no5w1KbfuipfJ1z0hmHACQYQQVOFyRMqXkw5mTldrHL70hR/b8ymwDADKFoAKHenlof3miSWOlFl6+pthsNmYaAJBpBBU4hI+vr3y0c71S2zR7vswYEM0MAwDuGkEF96xYubLy3jdfKrXY/7wqx3/bz+wCAO4JQQX35I1Rw6Rc/VCl1q1cCLMKAHAIggruiq+/n8TuWKfU1kybLvOGj2JGAQAOQ1BBpj1cpaJ0mTRWqQ1r8pKc/ONPZhMA4FAEFWRKh3GjpHTN6kqNrR4AgLP4ikYHDx40L1u98ejZs6fOlpAOvyxZzPfquTGkLJ8wmZACAPDsFZV+/frJhAkT7OOUlBSt/eBmpUKqSsfxnyi1IY2fl9NHjjJdAADPDipGMElKStLdBtLRddp4eahCOaXGVg8AwCu2fgy9evWSU6dOyfbt2yU8PFz8/Pxu+/kBAQESFBSkHHA8/8BAc6vnxpCyeMw4QgoAwHtWVD755BMzoJw+fVpq1qwp0dHRUqhQIenWrVu6fyYiIkIGDBjg0j69TdnQ2tL+01ilNqhBMzmbdEJbTwAA7+QjIg59ExYjbBirJLdTpkwZ+fXXm9+grm3btjJu3DjJkSOHpKampruiEhgYaB8bKyqJiYkSHBzM+S0OEP79NCn0SAmlxlYPAMDRjNfv5OTkO75+Ozyo5MuXT/LmzXvbzzlw4IBcuXLlpnrZsmVl9+7dUrp0adm3b59Dv1DcXkC2bBK9eaVSmx/7ifw45VumDgDgcBl9/Xb41o9xvolx3I2KFStKWlqanDjBFoMrlX+6nrQZMUSpDQh7VlL+Ou3SPgAAsMw5KjVq1JDq1avLqlWrzCQVEhIiH3/8sUybNk3Onj2rqy2v02fJ95KnSCGlxlYPAEC8PahcvnxZWrVqZZ4Ya5xzYtz8zQgqI0eO1NWSV8maI7sMSViu1GYNipGEGXO09QQAgGWCyo4dO8xVFLhelecaSevoSKXWv04jOX/2HA8HAMBStN/wDa41aF283Jcz2D6+fOGC9K5en4cBAGBJBBUvkS04WAavj1dq3/UbLFvmLtTWEwAAd0JQ8QI1XmgmL0aq97bpW+tpuZjM5dwAAGsjqHi4mO1rxS/LPw/zuRMnZWD9plp7AgAgowgqHipHntwS9eMipTa1ez/5aYl6pQ8AAFZGUPFAdV5pKc17faDUjBNmjRNnAQBwJwQVD2O84/GNThw8JMObttLWDwAA94Kg4iGC788vkSvmK7W493rKLyvXaOsJAIB7RVDxAPXavybPvt9ZqfWqGiZXLl3W1hMAAI5AUPGwrZ7Du/fKqFbttPUDAIAjEVTcVO7CBaVvvPq+PBPe/kD+u26jtp4AAHA0goobatj5TXn67fZKrWeVULmamqqtJwAAnIGg4uZbPb9v2S5j23XR1g8AAM5EUHET+YoVlYiFM5Xa5+26yP4t27X1BACAsxFU3ECTbl0l7I3WSq1HpTqSdvWqtp4AAHAFgorF9Zj3rRR4+CH7eM+P6+XLd8K19gQAgKsQVCwqZ4H80n+5egO3Ma92kD927tLWEwAArkZQsaDqzzeVlgMi7OPkk6dkYINmYrt2TWtfAAC4GkHFYnrO/07uL/6gfTx3+ChZO2261p4AANCFoGLhG7gNbfyC/HUkUVtPAADo5qu7AYjUevkFJaScOnxEwsvXJKQAALweKyoa+fj4SN+lcyRXwQL22qxBMZIwQ11ZAQDAWxFUNMn7QFHpvUi9gdugp5rL2eNJuloCAMBy2PrRIKxNayWkHPttv3QrF0JIAQDgX1hRcSEfX1+J+nGRZM+V016b3m+IbJ67wJVtAADgNggqLmJccmxcenyjqHpNzHukAACAW2PrxwUadHhDCSmHft5tbvUQUgAAuD1WVJzI199PhiaskCxZA+21r3tFyvaFS535zwIA4DEIKk5SqFQJCZ89TalFhj0jf/91xln/JAAAHoetHydo9E4HJaT8vnmbudVDSAEAIHNYUXEgP39/idmxVqlN/rC3/LxslSP/GQAAvAZBxUGKli0tH0yfpNT612kk58+ec9Q/AQCA12HrxwGadOuqhJS9azeYWz2EFAAA7g0rKvcyeQEBMnzbj0rtq67dZffqdff4sAAAAPO1lmm4O8XKPybvfT1RqfWt9bRcTE5hSgEAcBC2fu5Ci97dlJBinCxrbPUQUgAAcCxWVDLBuHHbsC2rldr4ju/Lrxs2OfhhAQAABoJKBhWvXEHemfyFUutdo75cPn+BZxIAAE5CUMmAllG9pfp/mtjHW39YLN/2HuisxwQAAPwPQeU2ArJlk+jNK5Xa2PbvmHeaBQAAzkdQSUdQ3jwyYPVCpRZRrZ6kXrzogocFAAAYuOonHRUa1rP/PmHWXPOqHkIKAACuxYpKOrYtiDd/PfTzHjn8yx5XPiYAAOB/CCrpMO6Jsu6bWel9GAAAuABbPwAAwLIIKgAAwLIIKgAAwLIIKgAAwLIIKgAAwLIIKgAAwLIIKgAAwLIIKgAAwLIIKgAAwPuCSu/evWX9+vVy/vx5OXPmzC0/54EHHpAFCxaYn5OUlCQxMTHi5+fnrJYAAICbcdot9AMCAmTmzJmSkJAg7du3v+njvr6+snDhQjl+/LjUrFlTChUqJFOmTJErV65Inz59nNUWAABwMzZnHm3atLGdOXPmpnqjRo1sV69etd1///32WseOHW1nz561ZcmSJcN/f1BQkM1g/Orsr4WDOeA5wHOA5wDPAZ4D4pA5yOjrt7ZzVEJCQmTXrl1y4sQJey0+Pl5y5swpjz322G1XaoKCgpQDAAB4Jm3vnlywYEHzvJQbXR8bH0tPRESEDBgw4KY6gQUAAPeR0dftTAWV6Oho6dWr120/p0yZMvLrr7+Ksxg9jBw50j42zm0x/r3ExESn/ZsAAMB5gSUlJcUxQWXEiBEyadKk237OgQMHMvR3GSfRVqtWTakVKFDA/rH0pKammsd1xhdXpEiR236RmZksI/A46u/zBMwJc8LzhO8dfp7w89WZrzFHjx697edkKqicOnXKPBzBuBrIuLonf/78cvLkSbP21FNPyblz52TPnj2Z+rvu9EVmlhFSCCrMCc8Tvnf4eeIc/IxlPq7LyGut085RMe6RkidPHilWrJh5b5QKFSqY9d9//928b8rSpUvNQDJ16lTp0aOHeV7K4MGD5bPPPlNWTAAAgHdzyqVWcXFxtlsJDQ21f06xYsVsCxcutJ0/f9524sQJW2xsrM3Pz0/bpV9c6syc8Dzhe4efJ/yM5TVHrHYJtvYGLHMEBATYIiMjzV9192KVgzlhTnie8L3DzxN+vorG1yGf//0GAADAcnhTQgAAYFkEFQAAYFkEFQAAYFkEFQAAYFkElXTMmzdPDh06JBcvXjRvKDdlyhTzdv3e6sEHH5SJEyeadx6+cOGCeT8c4z2XsmTJIt6sd+/esn79evPeQGfOnBFv1LlzZzl48KD5vbJx40apWrWqeLM6derI/Pnzzbtc22w2adasmXgz421XNm/eLMnJyeb7uc2ZM0dKlSol3qxTp06yc+dO8wanxrFhwwZp1KiR7rYsi6CSjlWrVknLli2ldOnS8vzzz0uJEiVk1qxZ4q2M93Dy9fWVjh07mu9u/cEHH5jfbEOHDhVvZryb98yZM2Xs2LHijYzvEeO9t6KioqRy5crmD1/jXdCNO057q+zZs5vz0KVLF92tWEJoaKh5I88aNWqYdx83/nNj3PDzvvvuE2915MgRM8BVqVJFnnjiCVm5cqX5n+OyZcvqbs2ytN+rwx2OJk2a2NLS0mz+/v7ae7HKER4ebtu/f7/2PqxwtGnTxnbmzBntfbj62Lhxo23MmDH2sY+Pj+3IkSO2nj17au/NCoehWbNm2vuw0pEvXz5zXurUqaO9Fysdf/31l61du3ba+xALHqyoZEDu3LnllVdeMZfnrl696vzo6CZy5swpp0+f1t0GNDH+Z2z8j3D58uX2mrHVYYxDQkJ4XJDuzw0DPzv+n7FS/dJLL5krccZ74OFmBJXbGDZsmPz999/mN5TxnkXevtd8I2MrrGvXrjJu3DjdrUCTfPnyib+/v3newY2MsfHeXcC/+fj4yKhRo2TdunWye/dur56gxx9/3HxDvsuXL8sXX3whLVq0kL179+puy5K8KqhER0eb/+O73WGck3JdbGysVKpUydxXTUtLM0+o9fY5MRQuXFiWLFlinpthnGDrae5mTgDcmXGuivEC3apVK6+frl9//VUqVqwo1atXN89xmzx5sjz66KNePy+34lW30Df+B5g3b97bfo5xVcuVK1duqhcpUsQ8AcpY0jaubPDWOTGufFq9erU5B2+88Yb5ou1p7uZ50qZNG/N/isY2oTdt/RhXgL3wwgvmiYDXTZo0SXLlyiXNmzcXb2d8fxjzcOP8eKsxY8aYq9J169aVP/74Q3c7lrNs2TLZv3+/eZECVP7iRU6dOmUed7uPaAgMDBRvnRNjJcW4Gmrbtm3Stm1bjwwp9/o88SZGUDOeC/Xr17e/EBtL+8b4008/1d0eLBZSjK2NsLAwQsptXmM87fXFUbwqqGRUtWrVzHtBGPuoxr0xjPMxBg0aZN47xFtPdjJCirGSYtxbJjw8XLn89N/nKHiTBx54QPLkyWOew+Tn5ycVKlQw68Zzxbi3iqczLk02lqy3bt1q3ivj/fffN08KjIuLE29lfP0lS5a0j4sXL24+L4xz3Q4fPizeuN3TunVrczXFOCejQIECZt24f8ilS5fEGxm3dVi8eLH8+eefEhQUZM6PEeIaNmyouzXL0n7pkdWOxx9/3LZixQrbqVOnbBcvXrQdOHDA9vnnn9sKFy6svTedl9+mR3dvOo+4uLhbzkloaKj23lx1dOnSxfbHH3/YLl26ZF6uXK1aNe096TyMx/5WjOeK7t50HOkxfqbo7k3XMXHiRNvBgwfN75mkpCTbsmXLbA0aNNDel1j08KpzVAAAgHvxqqt+AACAeyGoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAyyKoAAAAsar/AwVad+4GlZroAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "\n", "plt.plot(xs, ys)" ] }, { "cell_type": "markdown", "id": "617fb3d9", "metadata": {}, "source": [ "So, as of this notebook cell, we've data for regression and we've the PRNG keys for Jax. What's next then? THE MODEL!\n", "\n", "## Model\n", "### Model definition\n", "\n", "The python function should be pretty self explanatory. We pass in the parameters and the input array and the function will predict some output based on the given parameters. On academic terms, this function serves as the \"forward\" pass." ] }, { "cell_type": "code", "execution_count": 6, "id": "586344e4", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.556494Z", "start_time": "2023-02-12T22:45:58.554534Z" } }, "outputs": [], "source": [ "@jax.jit\n", "def lin_reg_model(params, x):\n", " w, b = params\n", " \n", " y_pred = w * x + b\n", " return y_pred" ] }, { "cell_type": "markdown", "id": "ec87a3bf", "metadata": {}, "source": [ "### Initial parameters" ] }, { "cell_type": "code", "execution_count": 7, "id": "52359e5a", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.653635Z", "start_time": "2023-02-12T22:45:58.557296Z" } }, "outputs": [ { "data": { "text/plain": [ "Array([1.2956358, 1.3550104], dtype=float32)" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def init_params():\n", " return jax.random.normal(subkeys[1], shape=(2, ))\n", "\n", "params = init_params()\n", "params" ] }, { "cell_type": "markdown", "id": "ece77a2c", "metadata": {}, "source": [ "### Error / Objective / Loss\n", "\n", "To be honest, academicians need to sit together and fix a single name for the same thing so people get less confused. Just get your thing together people! Anyways, so a error / loss / ........... just say loss function, we have to find one and define it. \n", "\n", "[Mean Squared Error](https://en.wikipedia.org/wiki/Mean_squared_error) is a pretty simple \n", "loss function for discrete estimation tasks such as linear regression." ] }, { "cell_type": "code", "execution_count": 8, "id": "d09295a6", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.710608Z", "start_time": "2023-02-12T22:45:58.655199Z" } }, "outputs": [ { "data": { "text/plain": [ "Array(10.507752, dtype=float32)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# jit, because speed\n", "@jax.jit\n", "def compute_loss(params, x, y):\n", " y_pred = lin_reg_model(params, x)\n", " \n", " # mse error\n", " error = jnp.mean((y - y_pred) ** 2)\n", " \n", " return error\n", "\n", "# test\n", "compute_loss(params, xs, ys)" ] }, { "cell_type": "markdown", "id": "2e73748d", "metadata": {}, "source": [ "## Gradient Descent\n", " \n", "\n", "To update using gradient descent, we need the gradients. `jax.grad` will give us the gradients, but I also want to keep track of the loss values for each iteration. Does jax have anything for this? Yes. `jax.value_and_grad`. \n", "\n", "So just to recall, a gradient descent update roughly looks like this in terms of pseudocode:\n", "```python\n", "params = params - learning_rate * gradients\n", "```\n", "\n", "This can be done in two ways in jax. Either via using `tree_map` or just simply like the pseudocode above. I've initialised the parameters as an array. What if you've a dictionary for params? In that case, jax forbids using the regular pythonic way to update dict values recommends using `tree_map`." ] }, { "cell_type": "code", "execution_count": 9, "id": "558add86", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.713676Z", "start_time": "2023-02-12T22:45:58.711533Z" } }, "outputs": [], "source": [ "# keeping the learning rate as 0.01 here. feel free to experiment with other values\n", "\n", "@jax.jit\n", "def update(params, x, y, lr=0.01): \n", " # get loss value and also gradients\n", " values, grads = jax.value_and_grad(compute_loss)(params, x, y)\n", " \n", " updated_params = params - lr * grads\n", " # alternative\n", " # updated_params = jax.tree_map(lambda p, g: p - lr * g, params, grads)\n", "\n", " return values, updated_params" ] }, { "cell_type": "markdown", "id": "8197b6a9", "metadata": {}, "source": [ "## Train" ] }, { "cell_type": "code", "execution_count": 10, "id": "ba2459dd", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.801595Z", "start_time": "2023-02-12T22:45:58.714632Z" } }, "outputs": [], "source": [ "epochs = 100\n", "\n", "losses = list() # keep a list to plot later\n", "\n", "\n", "for e in range(epochs):\n", " values, params = update(params, xs, ys)\n", " \n", " # append the loss\n", " losses.append(values)" ] }, { "cell_type": "markdown", "id": "c29c3787", "metadata": {}, "source": [ "Let's check on the updated params. They should be as close as possible to 4 and 3 (compare with the data generation function)." ] }, { "cell_type": "code", "execution_count": 11, "id": "deff959f", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:58.805123Z", "start_time": "2023-02-12T22:45:58.802591Z" } }, "outputs": [ { "data": { "text/plain": [ "Array([3.6967926, 2.7618957], dtype=float32)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "params" ] }, { "cell_type": "markdown", "id": "f6cbb354", "metadata": {}, "source": [ "Looks quite close. How about the loss values though? Did the overall loss decrease well over the iterations (epochs)?" ] }, { "cell_type": "code", "execution_count": null, "id": "af960baa", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:59.080829Z", "start_time": "2023-02-12T22:45:58.805997Z" } }, "outputs": [], "source": [ "plt.plot(jnp.arange(epochs), losses, c=\"r\")" ] }, { "cell_type": "markdown", "id": "10d65608", "metadata": {}, "source": [ "## Comparing the actual and predicted function\n", "\n", "We have the estimated parameters and now can use these to generate outputs using our linear regression model. (Estimating parameters is basically estimating a function). However one question lingers, how much of a deviation does our predicted function/model/parameters show?" ] }, { "cell_type": "code", "execution_count": 12, "id": "eff57421", "metadata": { "ExecuteTime": { "end_time": "2023-02-12T22:45:59.204191Z", "start_time": "2023-02-12T22:45:59.081727Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGdCAYAAAA8F1jjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAOtlJREFUeJzt3Qd4VFX+xvGXFhAIHWkCIqCoKIhKs2BBZd1dcV0F1N1FsevfigVwXYKgWAFFRQQF0bXAKooVRbADYkNFBaVJL9J7cf7PubkT7pxMQiaZmXtn5vt5ntHcMyUnd0Ly5txzfqeUpJAAAAACqLTfHQAAACgIQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAARWWaWB+vXra/PmzX53AwAAxCA7O1vLly9P76BiQsqyZcv87gYAACiGBg0aFBpWUj6ohEdSzBfKqAoAAKkzmmIGGvb3uzvlg0qY+UIJKgAApBcm0wIAgMAiqAAAgMAiqAAAgMBKmzkqhSlVqpSqVavmTNwxHyO1hEIhZ/7Rhg0bnI8BAJkj7YNK7dq1dcUVV6hFixZ+dwUl9PPPP2vUqFFas2YN5xIAMoQZXkjpP1HNKMmmTZtUpUqVfKt+ypYtqyeeeEJbtmzR+PHjtXr1au3du9e3vqJ4ypQpowMPPFDdunVT5cqVde2112rPnj2cTgBI09/fGTOiUq9ePVWoUEEPPfSQ5s2b53d3UAILFizQunXr9O9//1t169bV0qVLOZ8AkAHSejJt6dK5X97OnTv97griIPw+mhEWAEBmSOugAgAAUhtBBQAABBZBBTEZM2aMJk6cyFkDACQFQSUN9e/fX998843f3QAAoMQIKgAAIKo/S7pdUlX5J+OCSkWfbrE666yz9Mknn2j9+vVau3at3njjDR1yyCF59zdo0EAvvPCCfv/9d6dOzKxZs9S2bVv17NlTOTk5at26tVPF1dxMW+PGjZ2PW7VqlfcaVatWddo6deqUt0pq9OjRzlLgbdu2OQXWbrjhhricdwBAapku6U1J90s6x8d+pHUdFZsJDFt9+tyVJG2L5fGVKmnIkCH67rvvnCJnd999tzM3xASQihUr6qOPPtKyZct0zjnnaOXKlWrTpo0TNF5++WW1bNlSXbp0UefOnZ3X2rhxo+rUqbPfz2meb+qTXHDBBU4A6tixo5566imtWLFCEyZMKMFXDwBIFfUkLbfa/JyZmFFBJZW8+uqrEce9evVyRlaOOOIIJ0CYrQGOP/54Z8TFmD9/ft5jzQiLqdy6atWqmD6neY4ZjQlbtGiROnTo4FSEJagAQPrrKWms5/h3SQdK+sPHPmVUUNnmjmz49blj0axZM2cUpV27dqpVq1Ze8bpGjRo5oypmsmw4pMSTKU9vQpH5PAcccICysrL07bffxv3zAACCZbakoz3Ht0l6SP7LqKBSnMDgFzMnZfHixc6GisuXL3eCypw5c5zgsH379phf748/cvOwd/focuXKRTyme/fuznYDvXv31vTp0529F2677TYnLAEA0lNDSb9ZbYdK+kXBkHGTaVNBjRo1nN2eBw0apKlTpzqTWqtXr553v5m3YkZVvG1eu3btyldmPrzjsNn/KMy8htcJJ5ygzz//XCNGjHBGUczlpKZNm8b5qwMABMWVVkhZ6gaDoIQUg6ASQOGVPldeeaUTFE499VRnYm3Yiy++6Eygfe2115z5Kk2aNNF5552n9u3b580tMW1mhU/NmjWdUZgdO3Y4oyR9+vRxQtDJJ5/sBCGvX375Rccdd5zOPPNMNW/e3Ln0ZObBAADSSyk3jIz0tN3gjq6EFCwElQAyS4Z79OihY489Vj/88IOGDh3qXIIJ2717txMmVq9erbffflvff/+9E0D27t3r3P/KK6/o3Xff1bRp05zAc+GFFzrtZu5J2bJl9dVXX2nYsGHOTsReI0eOdCbxmpVDM2fOdELOE088keSvHgCQSE3cybHNPG2m+MXwAJ/2UCrfsrOzQ4b5v31f48aNQ+PGjXP+73c/uZX8HPB+8n3EvyO+B/geUInOwfXmb2HPba4UKhXA39/eW8ZNpgUAIBMv9SwxxUKt+SmjFHwEFQAA0lhzSfOstobuxNlUwBwVAADS1G1WSPnWHV1JlZBiMKICAECaKS1praTqVtXZcUo9BBUAANLIEZLmWG31Ja1QauLSDwAAaeIuK6R87l7qSdWQYjCiAgBAiisraaukLE9bD0kvK/URVAAASGFHuxsKepkdj3M3TimhgyXVMnu3mP1Z5Asu/UALFy7UjTfeGFEZt2vXrkk/M/3793d2hQYAFM09Vkj5wL3UU+KQYl7kGkmXSPqLpMP9e0cYUUE+devWdfYbKmq4OPfcc3XMMcdwJgEgScpFGeA4T9LEeLy4qQp3hdX2s3xDUEkT5cqVc/YAiodVq1bF5XUAAPF3rKQvrbaaktbF48XNxJYWnuN17iZAppi9T7j0E1BmQ8Hhw4c7tw0bNmjNmjXObsbeyzVmU8Fnn31WGzdu1FNPPeW0n3DCCfr444+1bds2/fbbb3rkkUdUsWLFvOfVrl1bkyZNcu5fsGCBLrroonyf277006BBA73wwgv6/ffftWXLFs2aNUtt27ZVz549lZOTo9atWzvPMTfTZlStWlWjRo1yNk40/fvggw909NHmSuo+d9xxh7ML9KZNmzR69GhVqFAhIecSANLFECukvOlepSlxSKkhKccKKS9JetTfkJKZQaWcT7diML/09+zZ44QCM4fklltu0eWXX553/6233qrZs2c7l10GDhyoQw45xNk12eyebEJB9+7ddeKJJ+qxxx7Le87YsWPVsGFDnXrqqTr//PN17bXX6sADzbSr6CpVqqSPPvrICSvnnHOOWrVqpQceeEClS5d2dll+6KGHnB2ezeUiczNtxoQJE5zX/dOf/uTsAv311187YaV69dzyQxdccIETcvr166fjjjtOK1ascPoCAMivvJsXbva0/VnSX+NxsrpIusFzbD7RIH8v92TupR8TGO70ccZTjFdmlixZoptvzv22nDdvno466ijn2Iw+GFOnTtWQISZf5zIjGP/973+dURTj119/1Q033OAEjWuuuUaNGjXS2WefreOPP15ffpmbyS+77DL9/HPB341mxMWMwpjnhOetzJ8/P+9+M8JiwpT3cpEZ1THhygSVXbtyr6LedtttzlwWE45MP2+66SY9/fTTeuaZZ5z777rrLnXu3JlRFQCwdHDroXhVk7SxpGeqkltjX9YQjX1dyWeZN6KSQmbMmBFxPH36dDVv3twZzTDCYSPMjHZccskl2rx5c95t8uTJKlOmjJo0aaLDDz/cmcfy1Vdf5T1n7ty5hU6cNZd1zEqcok6uDfejcuXKzqUib19MH5o2beo8xvRl5syZ+b4+AMA+T1ghZYJ7qafEIaVjlJByf/BCSuaNqOx2Rzb8+txxtnWrKe+zjwkHI0eO1KOPmouKkcx8lUMPPTTmz7F9+/aYn2P6YS7lnHLKKfnuM/NtAACFMzMLI3/CS2dImlLSE2cqwvWz2j42Q/TBfUcyK6gkKDAkSrt27SKO27dvr19++UV//PFH1MebeSBHHHFExKUZL3OJx6wOMnNGwqMxJryE541E89133znzYsxjoo2qmEs7ZsTG7oeZr2IuCS1evDjq6/7000/O1/fcc89FfH0AkOlOlvSR1ZZtLrWX9IWPkvR3q21oPIZnEotLPwFm5pQ8/PDDTpjo0aOHrr/++rz5J9Hcf//96tixo7NSyFx+adasmTMB1hyH57m88847zqiLmUPSpk0bZ76LWQFUkBdffNFZmfPaa685r20u35x33nl5oWLRokVOm/l8NWvWVFZWlqZMmeJcxjHPOeOMM9S4cWN16NBBgwYNckKSYb6OXr16OZeqzOUsM7H2yCOPjPs5BIBUMsYKKc+6l3q2lPQ3/W1WSJntrvIJeEhJeFA56aSTnKWwy5Yti1rtdMyYMXnLWsM384sUucaNG6cDDjhAX3zxhR5//HHnl3t4GXI033//vTp16uQEm08++cSZW2KWNC9fvjzvMZdeeqlzbCbYvvrqq87rmSXEBTFzWs4880znMW+//bbzOfr06aO9e/c695sVRmalkVlOvXbtWl144YVOu5m0a5ZJm/fYBKSXXnrJCSzhSbfjx493ViqZFURmzoy5b8SIEbz1ADJSZXexjSkEG3aKdVwsTST9x504GzYiXpXhkieUqFuXLl1CAwcODJ177rkho2vXrhH3jxkzJvT222+H6tSpk3erVq1aTJ8jOzvbeW3zf/u+xo0bh8aNG+f8P5FfZyJu06ZNCw0dOtT3fgTplsrvJzfOAd8DfA8U9D3Q2ZSvsm4V43G+LldIOZ5br2C9B4X9/vbeEjpHxfylbW6F2blzJ5VQAQAZydRU6+45ftLdYqdE6kq62moba67VKyX5PpnWrAwxlwPMRE1TF8RUW123ruAae2YORPnypvRNruxsM8UIAIDUUdWsgoyyYrjERRrOl9TSc7zZnTAbfQ1GSvA1qJjRFjNPwpSDN/U17r33XmeOipl4WdDKlr59+zoTL9OdqRwLAEg/Z0t6y2o7QNKOkrxoNUk3WW2m6MocpTxfg0q43LphyrCbpbBm/xkzymJGV6IZPHhwRDVWM6JiJusCABB0k6yy98OssvjF0lnSiSWvhh5Uvl/68TIjK2bzPbOstqCgYup2hMuyAwCQCsyef79bbceXtBBsRUm3W21mWmhkUfOUF6igYja+M7U4TFXTeDDLnY2yZQP1ZaKYwu9j+H0FgFRwbpTVwGamZYn+5G7rXkPyejBKOds0kNDf4GbnXTM6EhYuDGYmy5pb//79nTocpqCYmaNiamqYjfTM/jTxYPaaMVq0aFFgtVakDvM+GqZeCwCkAlPy/nTP8eAoFexLvLnu55LeU9pKaFA57rjj9OGHH+YdDx1qph5LY8eOdXbzPfroo9WzZ09Vq1bNKUL23nvvObvoxuvSjtkLx3z+bt265ZWQN2XdkXojKSakmPfRvJ+FVdIFgCCoLckupdnaLQhbbIdba5kNU6y86HvGpiRTmTelx9HNZNpNmzapSpUqzg69tlKlSjnVWKNtkIfUYkJKuJoxAARVd7c+SthOt/LsnpLUkL/RXdMc9qMp8S2l8+/vjAkqYRUrVlStWrWc4ILUYoKJudzDSAqAoPvMrYcS1l/S3SV5wcZm7xOrbaSk+EzlTInf3xkzy9T8kvvtt9/87gYAIA3VjZIdjnQHPoqtp7tXT5j5BE+l+vBC7DImqAAAkAj/NJvIeo5NxdlaknK3bi3mBJfrrLbnJGXomhCCCgAAxfStpFae4zskPVCSs9lV0jGe4x3usuNip57UR1ABACBGB0laYrUdKumX4p7JKpJusdomlnSZUHogqAAAEIPLJY3yHC+X1LAk+/6ZRan2wtTB7nIhEFQAACique7ISZhZNfxocU9fBUl9olSI+5T3w4sRFQAA9uNgsx+d1XZIlLYiayPpHKvtIUlbeCtsBBUAAArxf5KGe47NPJTDirtKuKw7iuL97TtL0lu8BQUhqAAAEIUpD7rYnX8SdrVbb61YzDWji6y24VG2VUYEggoAAJZmUVbwNIqy0qfIiec6t7hK2DxJL3Dai4KgAgCAR293ukjYd1atlJjXMZtlQl6jJS3llBcVQQUAAHfvvzWSanjOxiWSni3u2blYUnPPsXnxJzKvBH5JEVQAABnv8Cj78tQv7t5/NSVdb7W94F7uQbECJAAAGetOK6TMcKeVFCuknG2FFFP6fhAhpSQYUQEAZOwvwM1u3bWwCyW9VJwXqyzpVqttkqSvS9ZHEFQAABnoKHeSrFcdSauL82InSDrDarvP3VAQJcaICgAgowxyL/eETZN0WnFeqLykvlbbh+4NcUNQAQBkhHKSdlltf5f0anFe7GhJ51ltQyRtKn7/EB1BBQCQ9o6V9GWUxTnrYn2hMpJusya2fCPp9ZL3EdGx6gcAkNYeskLKW+6qnphDSlNJd1khxdRFIaQkFCMqAIC0VD7KfNa/FGf/P5NqrpRUz9O2sCSV4BALggoAIO20c+uheFWXtCHWFzLh5CqrbYy7WyGSgks/AIC08rgVUia4gyIxh5RuVkgxL3A3ISXZGFEBAKSFAyRts9rOlPR+rC9khl5utNrGR6mxj6QgqAAAUt5Jkj622qq4lWdjYpJNR6vtHkm7S9Y/FB9BBQCQ0p6RdKnn+DlJ/4r1RSpKut1qe1vSFyXvH0qGoAIASEmVJG2x2k4tTmHY9pK6WG0PRLmOBF8QVAAAKed0SVOiBJdtsZaq9dbSNz6N8sLwFat+AAAp5UUrSzzlruqJKaQcGSWkDCOkBBEjKgCAlGAmx26MsnHx57H+eX6zpGxP2w+S/hefPiL+CCoAgMD7kzu31V6ObFeeLdTBki6x2p6UtLLk/UPiEFQAAIH2mqSunuNHJN0U64v0ktTIc7xU0uj49A+JRVABAARS9SgbB7aVNCuWF6kj6RqrbZykBSXvH5KDoAIACJyu7kiKvcngrlhe5G+SWnmOt0p6WNIf8ekjkoOgAgAIlPckneE5vk9S31heoKo7YdbrFUnfx6d/SC6CCgAgEGpJWmO1HSPp21he5DRJJ1tt98Y6FIMgIagAAHx3gbvvX9hut6r9nqK+gFkCdEeUoZmY1i4jiCj4BgDw1adWSMmRlBVLSDkuSkh5kJCSLhhRAQD4oq6kFVZbS0lzYvkNdqdbljZshqR349dH+I+gAgBIun+4uxyHbZJUQ9Leor5AC0k9rLZHo6xnRsojqAAAkuorSW08x30k3V/UJ5vRk+vdVBP2s6SX4ttHBAdBBQCQFA3cgrBeh0maV9QXaCjpMqttlKRl8ekfgomgAgBIuMvdTBFm5qYcFEvttX9Kauo5XuXu0xOKbz8RPAQVAEBC/eyOnISZWmzDYimu8n9W238l/RK//iHYCCoAgIRoLGmR1dY0lm12/uIuPQ7b5U5mKfKMW6QDggoAIO6uk/SY53i+pOZFvVKTLam31fZarCVqkS4IKgCAuDGLcha7817DrnGnkxTJyW4ZfK/BknbGr49ILQQVAEBcmMs6v0a5/PNbUZ5cPsrOg1Mlfcybk+kooQ8AKLFbrJDygzu6UqSQ0jpKSHmYkIJcjKgAAEr01+4qd3FOWC9JY4ry5DLuHj1mYx9vNbg3eEOwD0EFAFAspor9T1GKui0vypPNzNqLrbbHJa3hzUAkggoAIGb9JN3jOf5CUruiPNFcD7paUh1rSZB34x8gWXNUTjrpJE2aNEnLli1TKBRS165d8z1mwIABWr58ubZt26b3339fzZo1S2SXAAAlYK7WbLVCysVFDSlmuKW/FVKeJqTAx6BSqVIlzZ49W9ddZ1bU53f77bfrhhtu0NVXX6127dpp69atmjx5ssqXN9O/AQBBcpSkPZIqetpM5nihKE82Ox1f4Tk2uxwPkLQk/v1E+gkl42Z07do1om358uWh3r175x1XqVIltH379lD37t2L/LrZ2dnOa5v/J+tr4cY54HuA74FM+x642/wc99ymFvW5NRRSjnVr4f/Xw02+n4Oi/v72bY5KkyZNVK9ePU2ZMiWvbdOmTZo5c6Y6dOigl19+OerzsrKyIkZcsrNNCUMAQCKUcyvXe50v6ZWiPLmLpPae45B7zcgMywBBr6NSt25d5/+rVpmFbfuY4/B90fTt29cJNOGbmf8CAIi/NlFCSq2ihJRKknKskPKme6mHkIJ0L/g2ePBgValSJe/WoIGZnQUAiKcH3JImYe+4C3Z+398TO0q6zWozGwl+yfuD4vHt0s/KlSud/9epUyfv4/Dxt98WvPPUrl27nBsAIP7MhfUdVts5RanBluWuWfb62C2DD6TiiMrChQu1YsUKnX766RHzTczqn+nTp/vVLQDIWO2ihJTqRQkpR0UJKUMJKUiBERWzPNlbF8VMoG3VqpXWrVunJUuWaNiwYfr3v/+tX375xQkuAwcOdGqqvPaa2c8bAJAswyX9n+f4FXfS7H7/1O3tzkkJmy1pYmL6iMyVsKVHnTp1CkUzZsyYvMcMGDAgtGLFCmdZ8vvvvx9q3rx5QpY3ceMc8D3A9wDfA/m/Bw6wlh2b25lF+V45JMqy4zp8j/E9prj//i7lfpCyzOUis/rHTKzdvHmz390BgJRxoqRPrLYqkvb7k/RySQd5js0Wyc/Ev39Ib0X9/c1ePwCQgUZLusxz/Lykf+7vSXXdfXq8xkpaFP/+AWEEFQDIIGY6yRar7TRJ0/b3RDNhpaXneJOkYZL+iH8fAS+CCgBkCBNIPrDaKrubDBaomqSbrLYJkubEv39AWhR8AwDE7r9WSBnlFnArNKR0jhJSTAl8QgqSiBEVAEhjZnLsxiiTaD8r7Elme+TbrbZ3Jc2If/+A/SGoAECa6uKWvvc6IEpRtwhtJZ0dpZ7+tvj3DygKggoApCFTc+1cz/Gjkm7c3zbJd1ptn0t6LzH9A4qKoAIAacSUvF8XZZBkVmFPOlxSd6vtEUnr498/IFYEFQBIE2bzwNejbDK4q7DlFGaYpaqn7UdJ4xPXRyBWBBUASANmrutZ1rSSOwp7QmNJl1ptIyWtSEz/gOIiqABACqslaY3V1kbSN4U9qafZJdZzbMLJU6m+oQrSFUEFAFLU+W7ttTBTJLaCpN0FPaG2pOustuckzU9cH4GSIqgAQAr6WNJJnuO7JfUv7AldJR3jOd4u6SFJexPXRyAeCCoAkELqSFpptbUsrFisqfh2S5S1y7MT0z8g3ggqAJAiLnZ3OQ7b7C5HLnBQ5BT35jVY0s7E9RGIN4IKAKSALyUd6znuK+m+gh5sJqr0sdqmSPo0cf0DEoWgAgABVl/SMquthaS5BT2hjVtQxcvMRdmSmP4BiUZQAYCA6iXpac/xKje4mNU9UX+am2GWMp42U472rcT3E0gkggoABNCPbmX7MDMfdmhBDz5U0kVW23BJvyeuf0CyEFQAIEAaSVpstTUrqNRJKbcuiqn6FjZP0guJ7SOQTAQVAAiIayU97jleJOmQggrGHiTpcqtttKSlie0jkGwEFQDwmRkYWehuv+MNLSMKW6fc3HNsaug/QQl8pCeCCgD4qKmkX602E1h+i/bgmpKut9pecC/3AGmKoAIAPrlZ0hDP8Ry3ymxUZ0tq6zne4xZSMf8H0hhBBQCSrLRbBt/sERh2maRnoj24sqRbrbZJkr5ObB+BoCCoAEASHSbpZ6utgaTl0R58gqQzrDYzirIjcf0DgoagAgBJYuqx3WvVY/NezclT3n2w14fuDcgwBBUASDBTLHajpEqetn9I+m+0Bx8t6TyrzUxk2ZTYPgJBRVABgAQyk2O/t9rquuXw86WZ29wNBcO+kfQ6bw8ym5nTBQBIgAFWSPnYrZmyKtoa5buskGLqohBSAEZUACDeyknaZbVdIOl/9gNNarlSUj1Pm6n89izvCRDGpR8AiKNjoqwcNsuQ19oPNOHkKqttTJSNfoAMx6UfAIiT+62QMtkdNMkXUrpZIWWDpLsJKUA0jKgAQAllSdpptXV167JFqC7pRqttvKQfeQuAghBUAKAETB2UmVZbDUnr7QeeKamj1XaPpN2cfqAwBBUAKKZHrT0CJ0YpgaKKkm632t6W9AWnHSgKggoAxOgASdusti7unJQI7d07vB6I8mQABSKoAEAMzPY7n1ptVe3CsWZ98p3Wg8yTpnCqgVix6gcAiugpK6T8113VExFSjowSUoYRUoDiYkQFAPbD7NGzxWo7XdJU+8++myVle9p+iFblDUAsCCoAUIhT7UAiqbKkrd6GgyVdYj3oSUkrObVASRFUAKAAz7m7HIc9Lely+0G9JDXyHC+VNJpTCsQLQQUALNn2vBNJJ9mTaOtIusZ60DhJCzidQDwRVADA4yxJ70YphbLd2/A3Sa08x+Y60MOS/uBUAvFGUAEA1ytWwbbhkm6w1yGbCbP2k77nFAKJQlABkPGqRSl5384uHnuapJOtB90raVfGnz4goQgqADLaX6NsHljBu8mgKUN7h/WA9yR9npz+AZmOgm8AMtbbVkh50C3glhdSjosSUsyDCClA0jCiAiDj1JS01mprI+kb70/GO93UEjYjyixbAAlHUAGQUc6XNMFqy5K0O3zQQlKPKNskr0tO/wBEIqgAyBgfSurkOR4o6T/hg1LuEp/qngf8LOml5PYRQCSCCoC0d6CkVVbbUe5WPI6Gki6zHjBK0rLk9A9AwQgqANLaRe4ux97abKYcyt5wwz8lNfU8YJW7T08ouf0EENBVP/3791coFIq4/fTTT353C0Aa+MIKKXe6Gwo6IaWWpBwrpJgHjyCkAEESiBGVH374QZ07d8473rNnj6/9AZDa6ke5anO4O+Ukr3jKsZ47TdG2+73DLACCIhBBxQSTVavsK8gAELtLJT3jOV4jqW54Gx6z22Bv6wmvSfqWMw0Ele+XfozmzZtr2bJlmj9/vp5//nk1bGhmtkWXlZWl7OzsiBsAyJ0c6w0pvd2JtE5IOTlKSBlMSAGCrpTfU8a6dOmiypUra+7cuapXr54zZ6VBgwZq2bKltmzZku/x5v6cHHNhOVKVKlW0efPmJPUaQJA0krTYamsmab75oLykvtadUyV9nLz+AcjPDDRs2rRpv7+/fQ8qtqpVq2rx4sW65ZZb9Mwz3r+N9o2olC9vfvLs+0LNaAxBBchM10h6wnNsAkuT8A+21pLOtZ7wsCT+pgFSJqgEYo6K18aNGzVv3jw1a2b+Hspv165dzg1AZivljpiYUBJ2XTi0lHH36DElZ8O+kvRG8vsJIA3mqHhVqlRJTZs21YoVK/zuCoCAOsSdd+INKQeHQ0pzSXdZIeUxQgqQqnwPKg8++KBOPvlkNW7cWB06dNDEiRO1d+9evfjii353DUAA3RSee+L6yR1dWWz+c62kiz13mgfmRNmBEEDK8P3Sz0EHHeSEkpo1a2rNmjX69NNP1b59e61dy08WAJF/VS2XVMfTdrmkp80HDSRdYZ0tc8cSziCQ6nwPKhdeeKHfXQAQcIdKmmu1HRQu6tbD3fE4zOxyPDxoywQApGxQAYDCmDmx91lzYo8zH9Rwdzv2eslbfhZAOiCoAAgks3BnvVtM1rt/4PPmgy6S2nvuMKMn95gy18nvJ4DEIqgACJwj3SqzXqYM/qpKkm6z7nhT0pdJ7ByAzFr1AwBeOVZI+cRd1bOqY5SQYjYSJKQAaY0RFQCB+WG0w73kE9ZN0gRTD6Wf9eCP3TL4ANIeQQWA70yl+2+sttqm/MlRkv5u3THUlLBOYucA+IqgAsBX97kre8Lek3RWaXenYzMnJWy2pIk+dBCArwgqAHxhrujstNrM/oGvm/r4/7LuGGEmqSSxcwACg6ACIOmOl/SF1VZd0oYr3CqzYb9Jyr+JOoAMwqofAEk1zAopr5tVPXWlDTlWSBlLSAHAiAqAJKkgabvVdrakd86X1NLTuMlNM2Z7ZAAZj0s/ABLOlED5zGrLriZtMVshe02QNIc3BMA+XPoBkFAjrZDyornU0zlKSDEl8AkpACyMqABIiIqStlptJ1eUPrndanxX0gzeBADREVQAxN0pkqZZbeXbSrvMpBSvByRt4w0AUDCCCoC4Gufuchw2opx07Z3Wgz53K7sBwH4QVADERba7YMfriMOln7pbjY9IWs9JB1A0BBUAJXampMme4z2lpawbpVBVT+OPksZzsgHEhlU/AEpkghVSbm4slfuPFVLM0h9CCoBiYEQFQLFUs67ghCRVvUTafLCncYWkp9w7AaAYCCoAYvYXSW94jufUllpeZz3oOUnzObkASoagAiAmb0r6s+e4bVdp1jGeBlMn/yFJezmxAEqOoAKgSGpKWus5XlJFanSL9aCJkmZzQgHED5NpAezXeVZIyTklSkgZTEgBEH+MqAAo1DS30qyxvoJUo4/1gCmSPuUkAkgMggqAqA6UtMpzPKqNdOU51oPMXJQtnEAAiUNQAZDPhZJecD/eUVaq0lfaXcbzgFmS3uLEAUg8ggqACDPNSh734zcOlc65yDpBwyX9zkkDkBwEFQCOepKWu+dibynpyOukubU8J2eeZ5gFAJKEoAJAl0ga456H6QdJHS+3TspoSUs5UQCSj6ACZLjvJB3lVrn/88XSO809d66R9AQl8AH4h6ACZKiGkn5zP55bU2pxvfWAF9zLPQDgIwq+ARnoKk9Iue5sK6TskTSIkAIgGBhRATJIKUm/SGpqNjauLNW/1XrAJElf+9Q5AIiCERUgQzSR9IcbUu47MUpIuY+QAiB4GFEBMsCNkoZJ2lReqtrXuvND9wYAAURQAdJ8yHSZpLqSnjta+pfZXdBriEkvPnUOAIqAoAKkqfamJoqkXWWkardJGyt47vxG0us+dg4AioigAqQhMyf2r5ImN5W6/NO609RFWe1TxwAgRgQVIM3+Qe82k2ZLSaX6W3culPSsTx0DgGJi1Q+QJjq5IeXNQ6Uydkgx9fEJKQBSECMqQBr4QNJppk5KTpQ7B5pdBn3oFADEAUEFSGFZknZK+qqeVMqUm/WaJukjnzoGAHFCUAFS1FmS3pV09DXS93WsOwe7CQYAUhxBBUhBMyUdWE0qdZN1x2xJE33qFAAkAEEFSCEHSNomqcf50sstrTuHStroU8cAIEEIKkCKOEfS2ApSqT7WHcslPeVTpwAgwQgqQAr4UdKLp0o1zBpkryclrfSpUwCQBAQVIMAqS1pbRqpwl3WH2Qb5bp86BQBJRFABAqqbpC6tpQrnWnc8L+lXnzoFAElGUAEC6DdJjXKk8fYdAySF/OkTAPiBoAIESFVJEw6RGv3LuuMNU9XNp04BgI8IKkBA9JT0v37SmabcrNcgSXt86hQA+CwQmxJee+21WrhwobZv364ZM2bo+OOP97tLQFJ9dqD0bI601RtSPpFk9u4hpADIYL4HlW7dumnIkCEaMGCA2rRpo9mzZ2vy5MmqXbu2310DEq6mpA6XSSdca91xv7vTIABkuFJ+T80zIyizZs3S9ddfn9uhUqW0ZMkSDR8+XPffb35aFy47O1ubNm1SlSpVtHnz5iT0GIiPf2RLz/eObCv7k7TnZc4wgPSXXcTf376OqJQrV07HHnuspkyZktcWCoWc4w4dOvjZNSChep2TP6ToEUIKAARqMm2tWrVUtmxZrVq1KqLdHLdo0SLqc7KyslS+fPmIRAakitpZ0pp+0jOetsprpS2P+dgpAAgw3+eoxKpv377OUFH4tmzZMr+7BBTJCSfmhhSvCqMIKQAQ2KCydu1a7dmzR3Xq1IloN8crV0bfwGTw4MHO9azwrUGDBknqLVCCf2U50medrfYcaQc5GwCCG1R2796tr776Sqeffnpem5lMa46nT58e9Tm7du1yJt14b0BQVW8p6T+Rbce95C47BgAEv+CbWZr87LPP6ssvv9QXX3yhm266SZUqVdKYMWP87hpQMjnSequpwgDpS0rgA0DqBJXx48c7NVPuvvtu1a1bV99++626dOmi1atX+901oHgamWU9kU03vyMNnSnt4JwCQGrVUSkp6qggSMr0lvZaC9FOu0eautuvHgFAMKVEHRUgbdTKvdTjDSnXz5SycggpAJDSl36AlPcPSc0im8Y/KHXb6leHACB9EFSA4qok6bbIpjN/lXY8L3XjrAJAXBBUgOL4k6R2kU0/PSYdtZbNjgEgnggqQCzKSbozsqnuZunJh6XDOZMAEHcEFaCo2rkjKR4fPyPd9pt0LmcRABKCoAIUZRF///zNf+Tk/gP6gzMIAAlDUAEKYzbx7hHZ9OL/pL0/sLYfAJKBoAIUJMp+PLvultr9IX3DWQOApCCoADazIfcVkU2Dp0h9Ps0dRUnpUs4AkGIIKoDX/7lVZj02DpZe3Jk7VQUAkFwEFcCoLunGyFPR62vp6UlSS0lzOEsA4AuCCtBd+YqgLH1YarCZURQA8BubEiJzHeBOmPWElA5LpFCONJ6QAgCBwIgKMtPpkk6KbPruCemo1dKhkn7xq18AgAgEFWTed/y/I5sq7ZK23Jv7MRNmASBYuPSDzNEmf0h5b1xuSLmHkAIAgcSICjK6BL65q4mkRX70CwCwXwQVpLdmkv4R2fTMa9Kl3+Z+zKUeAAg2ggrS112SykQ27Rgold8r3SnJnZYCAAgwggrST11JV0c23fWRdPe03I8PkrTMl44BAGJFUEF6uVJS/cimdfdJ1XfkfsylHgBILaz6QXqo4hZv84SU7j/kFm8zIeVmQgoApCRGVJD6/iapVWTTwmHSwRv2XQla5UvHAAAlRVBB6iovqW9kU8tV0vcj9h1zqQcAUhtBBanpZEmnRTZ9OVI6dkXux2Yu7UhfOgYAiCeCClJLGXfZscXMRQmrJen3pHYKAJAoBBWkjqMlnRfZ9MYL0l/m5X68UVI1XzoGAEgUggpSg2fEJGzvAKl0KPfjnpLGJb1TAIBEI6gg2A6WdElk0+NvSdfO2ndczR1NAQCkH4IKgut2SRUjm7YNkg7Yk/vxUkkNfekYACBZKPiG4KntXurxhJTen+dOmA2HlO6EFADICIyoIFgucS/3eKx5QKq1bd9xtqQtSe8YAMAPBBUEQ2VJt0Y2/Xme9OYL+45/knRE0jsGAPATQQX++7Ok4yOb5j0qNV+377irpElJ7xgAwG8EFfgnS1K/yKYqG6SNwyLbzFSV7UntGAAgKAgq8EcHSWdFNn32tNRxyb7jLyS1S3rHAABBQlBB8teZ/Sd/8x85kRsImgzzXjL7BQAIJIIKksfMhO0W2dR9vPTSj/k3Rd7F+wIAIKjAzxL4u++Wyv6x73iqpNN5SwAAHoyoILFM6djLIptqTpbWTo9sO0XSR7wXAAALQQWJc1P+7YzvuVfqZ13XKSfJLTgLAEAEggrir4akG6y2WVLorcimSW59FAAACkJQQXxdJOnQyKbmD0nztuRfnTyDcw8A2A+CCuKjorvbsdciacRY6eoo33R7Oe8AgCIgqKDkzpTU0Wp7XAqtiWwy2/ZczPkGAMSAoIKSfff822rbJrV+QPrGaj5W0tecawBAjAgqKJ7j3c0EvcZK4xZJ/4xSjDbEeQYAFANBBbExde77R2nOkTy12xyjJV3B+QUAlABBBUV3qLuqx+tVqe130kyr+ShJP3BuAQAlRFBBsUvga6D06l7pb1azd3NBAABKgqCCwtWXdKXVNlUq+7G022p+xC1GCwBAvBBUULBrJNWx2gZLJ+2UPraaD5M0j3MJAIgzggryqxZlaGS2pInSZLdsiheXegAAiWJWjvpm4cKFCoVCEbc77rjDzy7h/CghZYiUNTF3ibE3pNxLSAEApPuIyl133aVRo0blHW/evNnX/mSsCpL6WG3LJT0ldZb0vnXXISZoJrF7AIDM5HtQMcFk1apVfncjs50qqZPV9qSkldLn7gaCXlzqAQBkxKUfo0+fPlq7dq2+/vpr3XrrrSpTpkyhj8/KylJ2dnbEDcVUxl127A0ppmpbjlRhZe6lHm9IMdXyCSkAgIwZUXn00UedgLJu3Tp17NhRgwcPVr169dS7d+8Cn9O3b1/l5EQr6oGYtJZ0rtX2vKRfpb9IesO6q6GkpZxiAECSlYr3NiwmbJhRksK0aNFCc+fOzdd+6aWXauTIkapcubJ27dpV4IhK+fLl847NiMqyZctUpUoV5rcUVbScNyD3O+E7t6qsF6MoAIB4M7+/N23atN/f33EPKrVq1VLNmjULfcyCBQu0e7ddLkw64ogjNGfOHB122GGaN29eXL9QuDNg/2WdiUm52xpXkrTFusuMaw3hxAEAEqCov7/jfunHzDcxt+Jo3bq19u7dq9WrV8e7W+hnhqOs0zBI0h45K5InWHfVlcQUZwBAxs5Rad++vdq1a6dp06Y5SapDhw4aOnSonn/+eW3YsMGvbqWfOm6FWa9PJH2Q+6FZYnywdTeXegAAyvSgsnPnTvXo0cOZGGvmnJjibyaoDBnCxYa4ucydBet1v6TtUhVJG627rnFXJQMAoEwPKt98840zioIEMCu27YVTP0oan/vhPyQ9Z91dS9LvvBkAgIDxveAb4uwcSW2ibGu8PvdDE0ZqeO4yE2ipRAMACCqCSrrIcifMepk5zY/lflhd0jrr7ksljU1W/wAAKAaCSjo4Ubkb8niZ7ZOW5X54Re6WPRFMcGHKMgAg6Agqqb4Bwn8KL+hmyuaVs/YZbJCMvgEAEAcElVTVUrkFULxekvRz7oe1JdnVaHpIejlZ/QMAIA4IKmlWAt+4wZ0/65UdpfIsAABBR1BJJY3dGbBe70iaue/Q3g/B7KjUIhl9AwAgAQgqqaJ3lHXE90hyt0yqv2/ubB6zOfLryeofAAAJQFAJumqSbrLaZkh6d9/hHZLusx5SMbcALQAAKY2gEmQnSzrNantQ0taCL/V8Ken4ZPQNAIAkIKgEUXlJfa22XyU9HzldZZH1kD9FDrQAAJDyCCpB09qdXOL1sKTNkYt++lsPqWA2ekxG/wAASCKCSlCUcSebmFL4YV9JeiPyYfalng8lnZqM/gEA4AOCShA0l3Sx1faYu1ePq5mkX6yHnOoGFQAA0hVBxU+lJF0j6UBP23xJz+WfP3ur9dSsfSuTAQBIWwQVvzRwdwv0elrSksimHyUd7jl+U9Jfk9E/AAACgKDihx5Wudh1koZHTkAxOWap9bSOkqYnq48AAAQAQSWZargb8RSwkWDY5ZJGeY5XSDpI0h/J6CMAAAFCUEmWLpLae45Dbgn8PZEPM5nlMM/xTVE2GAQAIFMQVBKtkqTbrLY33RKyKryAW1NJCxLdPwAAAoygkkhmUsmZVtv9+Tfhuc5djexd+NM8Ss0UAAAyDUElEcza4X5W28eSpuZfnbxYUkNPm1mt/GRCOgUAQOohqMTbUZL+brUNlbQx/2Uds32PV6P8q5MBAMhoBJV4KS2ptzsnJWy2pIn5H2oe9pDn+HtJR8etIwAApA+CSjwcIulfVtsISavyZ5nVkmp62npJGhOXTgAAkH4IKiV1hVudLew3Sc/kf5ip7/aT1VbfrZECAACiI6gUV11JV1ttY6OsMZZ0p6RBnuOZVkkVAAAQHUGlOM6X1NJzvEnSsPylY8u6dx3gaTObJL9QrE8KAEDmIajEoppbKtZrgqQ50Rf/fGe11XHnqAAAgKIx8ztRFJ2jhJR7ooeUu62QMs2tmUJIAQAgNoyo7E9FSbdbbe9KmpH/oeUk7YpyleiVGN8UAACQi6BSmLaSzrbaHpC0Lf9D20j6ymqrJen3Qj8BAAAoDJd+CquN4g0pn0vKiR5SHrRCytvupR5CCgAAJcOISkHMdZywRyStz/+Q8pJ2WG3nSHqjhG8KAADIRVApyFxJAyXtjX53uyjTVKpL2lDgCwIAgFhx6acwBYSU4VZIecW91ENIAQAgvhhRicEBUaaonCXpvfi+JwAAwEVQKaITJX1itVWRtLmoLwAAAGLGpZ8iGG2FlOfcSz2EFAAAEosRlUJUkrTFajvNrTQLAAASj6BSALMvz0qrrbKkrYl/TwAAgItLPwW4wPPxU+6lHkIKAADJxYhKAZ53/z9T0qzkvR8AAMCDoFIAUxPlsYLuBAAAScGlHwAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAkHlBpV+/fvrss8+0detWrV+/PupjGjZsqDfffNN5zKpVq/TAAw+oTJkyieoSAABIMQkroZ+VlaUJEyZo+vTpuuyyy/LdX7p0ab311ltauXKlOnbsqHr16mncuHHavXu37rzzzkR1CwAApJhQIm89e/YMrV+/Pl97ly5dQnv27AkdeOCBeW1XXXVVaMOGDaFy5coV+fWzs7NDhvl/or8WbpwDvgf4HuB7gO8BvgcUl3NQ1N/fvs1R6dChg77//nutXr06r23y5MmqWrWqjjzyyEJHarKzsyNuAAAgPfm2e3LdunWdeSle4WNzX0H69u2rnJycfO0EFgAAUkdRf2/HFFQGDx6sPn36FPqYFi1aaO7cuUoU04chQ4bkHZu5LebzLVu2LGGfEwAAJC6wbN68OT5B5eGHH9bYsWMLfcyCBQuK9FpmEm3btm0j2urUqZN3X0F27drl3MLMF9egQYNCv8hYTpYJPPF6vXTAOeGc8H3Cvx1+nvDzNZG/Y5YvX17oY2IKKmvXrnVu8WBWA5nVPbVr19aaNWuctjPOOEMbN27Ujz/+GNNr7e+LjJUJKQQVzgnfJ/zb4edJYvAzlvMRVpTftQmbo2JqpNSoUUONGjVyaqO0atXKaf/111+duinvvfeeE0iee+453X777c68lEGDBunxxx+PGDEBAACZLSFLrcaMGROKplOnTnmPadSoUeitt94Kbd26NbR69erQgw8+GCpTpoxvS79Y6sw54fuEfzv8POFnLL9zFLQl2L53IDC3rKysUP/+/Z3/+92XoNw4J5wTvk/4t8PPE36+ysffQ6XcDwAAAAKHTQkBAEBgEVQAAEBgEVQAAEBgEVQAAEBgEVQK8Prrr2vx4sXavn27U1Bu3LhxTrn+TNW4cWONHj3aqTy8bds2px6O2XOpXLlyymT9+vXTZ5995tQGWr9+vTLRtddeq4ULFzr/VmbMmKHjjz9emeykk07SpEmTnCrXoVBIXbt2VSYz26588cUX2rRpk7Of28SJE3XooYcqk1199dWaPXu2U+DU3D7//HN16dLF724FFkGlANOmTVO3bt102GGH6e9//7uaNm2q//3vf8pUZg+n0qVL66qrrnJ2t7755pudf2z33nuvMpnZzXvChAkaMWKEMpH5N2L23howYIDatGnj/PA1u6CbitOZqlKlSs55uO666/zuSiB06tTJKeTZvn17p/q4+ePGFPysWLGiMtXSpUudAHfsscfquOOO09SpU50/jo844gi/uxZYvtfqSIXbX//619DevXtDZcuW9b0vQbndeuutofnz5/vejyDcevbsGVq/fr3v/Uj2bcaMGaHhw4fnHZcqVSq0dOnS0B133OF734JwM7p27ep7P4J0q1WrlnNeTjrpJN/7EqTb77//HurVq5fv/VAAb4yoFEH16tV18cUXO8Nze/bsSXx0TBFVq1bVunXr/O4GfGL+MjZ/EU6ZMiWvzVzqMMcdOnTgfUGBPzcMfnbkMiPV3bt3d0bizB54yI+gUoj77rtPW7Zscf5BmT2LMv1as5e5FHb99ddr5MiRfncFPqlVq5bKli3rzDvwMsdm7y7AVqpUKQ0bNkyffvqp5syZk9EnqGXLls6GfDt37tSTTz6pv/3tb/rpp5/87lYgZVRQGTx4sPMXX2E3Mycl7MEHH9QxxxzjXFfdu3evM6E208+JUb9+fb377rvO3AwzwTbdFOecANg/M1fF/ILu0aNHxp+uuXPnqnXr1mrXrp0zx+3ZZ5/V4YcfnvHnJZqMKqFv/gKsWbNmoY8xq1p2796dr71BgwbOBCgzpG1WNmTqOTErnz788EPnHFxyySXOL+10U5zvk549ezp/KZrLhJl06cesADv//POdiYBhY8eOVbVq1XTuuecq05l/H+Y8eM9Ppho+fLgzKn3yySdr0aJFfncncN5//33Nnz/fWaSASGWVQdauXevcinsd0Shfvrwy9ZyYkRSzGuqrr77SpZdempYhpaTfJ5nEBDXzvXD66afn/SI2Q/vm+LHHHvO7ewhYSDGXNk455RRCSiG/Y9Lt90u8ZFRQKaq2bds6tSDMdVRTG8PMxxg4cKBTOyRTJzuZkGJGUkxtmVtvvTVi+ak9RyGTNGzYUDVq1HDmMJUpU0atWrVy2s33iqmtku7M0mQzZP3ll186tTJuuukmZ1LgmDFjlKnM19+sWbO84yZNmjjfF2au25IlS5SJl3suuugiZzTFzMmoU6eO027qh+zYsUOZyJR1eOedd/Tbb78pOzvbOT8mxJ111ll+dy2wfF96FLRby5YtQx988EFo7dq1oe3bt4cWLFgQeuKJJ0L169f3vW9+Lr8tiN998/M2ZsyYqOekU6dOvvctWbfrrrsutGjRotCOHTuc5cpt27b1vU9+3sx7H435XvG7b37cCmJ+pvjdN79uo0ePDi1cuND5N7Nq1arQ+++/H+rcubPv/VJAbxk1RwUAAKSWjFr1AwAAUgtBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAAKKj+H1kYL0Xn2KgMAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(xs, ys, c=\"r\", label=\"actual\")\n", "plt.plot(xs, lin_reg_model(params, xs), c=\"g\", label=\"predicted\")\n", "plt.legend()" ] }, { "cell_type": "markdown", "id": "410db8cd", "metadata": {}, "source": [ "You're pretty good - [Revolver Ocelote](https://youtu.be/8xVfjkb-xVs)" ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.13" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 5 }