{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Lin Reg Optax" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Optax without Flax ? \n", "\n", "So in notebook number 07, I showed how you can train a simple linear regression model using jax. The whole optimisation process there was done manually. Although it was just an introduction, such manual optimisation isn't really recommended for larger projects. So continuing from notebook 08, what if you don't want to use flax, write everything from scratch using jax and now want to use optax? Let's use the same code from notebook 07 and replace the manual optimisation process with optax. " ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "\u001b[32m2026-05-07 19:30:15.101\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36msetup_backend\u001b[0m:\u001b[36m24\u001b[0m - \u001b[1mSetting up JAX to use the `mps` backend.\u001b[0m\n", "\u001b[32m2026-05-07 19:30:15.101\u001b[0m | \u001b[32m\u001b[1mSUCCESS \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36msetup_backend\u001b[0m:\u001b[36m26\u001b[0m - \u001b[32m\u001b[1mJAX Platform set to use the `mps` backend.\u001b[0m\n", "Platform 'mps' is experimental and not all JAX functionality may be correctly supported!\n", "\u001b[32m2026-05-07 19:30:15.232\u001b[0m | \u001b[1mINFO \u001b[0m | \u001b[36mutils.device\u001b[0m:\u001b[36mlog_device\u001b[0m:\u001b[36m14\u001b[0m - \u001b[1mXLA DEVICE: mps\u001b[0m\n" ] } ], "source": [ "# setting up device backend for apple silicon (mps)\n", "# if you have it that is.\n", "\n", "from utils import setup_backend, log_device\n", "\n", "setup_backend()\n", "log_device()" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import jax\n", "import jax.numpy as jnp\n", "\n", "masterkey = jax.random.key(0)\n", "masterkey, *subkeys = jax.random.split(masterkey, 10)\n", "\n", "\n", "\n", "f = lambda x: 4 * x + 3\n", "f(1)\n", "\n", "\n", "\n", "xs = jax.random.normal(subkeys[0], shape=(1000, ))\n", "ys = f(xs)\n", "\n", "\n", "\n", "\n", "@jax.jit\n", "def lin_reg_model(params, x):\n", " w, b = params\n", " \n", " y_pred = w * x + b\n", " return y_pred\n", "\n", "\n", "def init_params():\n", " return jax.random.normal(subkeys[1], shape=(2, ))\n", "\n", "params = init_params()\n", "\n", "\n", "@jax.jit\n", "def compute_loss(params, x, y):\n", " y_pred = lin_reg_model(params, x)\n", " \n", " # mse error\n", " error = jnp.mean((y - y_pred) ** 2)\n", " \n", " return error\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Enter the Optax" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "import optax\n", "\n", "optimiser = optax.sgd(learning_rate=0.01)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's take a look at the previous `update` function from notebook 07. \n", "\n", "```python\n", "@jax.jit\n", "def update(params, x, y, lr=0.01): \n", " # get loss value and also gradients\n", " values, grads = jax.value_and_grad(compute_loss)(params, x, y)\n", " \n", " updated_params = params - lr * grads\n", " # alternative\n", " # updated_params = jax.tree_map(lambda p, g: p - lr * g, params, grads)\n", "\n", " return values, updated_params\n", "\n", "```\n", "\n", "We no longer need the manual update. Optax will do that but we also have to pass in the optimiser and the initial optimiser state. Where's the problem with that? The `jit`. Optimisers aren't jit'able. So we have to rewrite the function a bit. Actually, let's just write one training function that takes in the data, optimiser and everything else and gives us trained params. (Funny that you're always training the params and not the model and yet we call it model training. xD) " ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "from tqdm.auto import trange\n", "\n", "\n", "def train_model(params, x, y, optimiser, epochs=100):\n", " # create the initial optimiser state\n", " # well this way, every time train_model is called, the params are trained with a new optimiser state\n", " optimiser_state = optimiser.init(params)\n", " \n", " @jax.jit\n", " def update(params, x, y, optimiser_state): \n", " # get loss value and also gradients\n", " values, grads = jax.value_and_grad(compute_loss)(params, x, y)\n", " # get updates\n", " updates, optimiser_state = optimiser.update(grads, optimiser_state, params)\n", " # now apply updates\n", " params = optax.apply_updates(params, updates)\n", "\n", " return values, params \n", " \n", " \n", " \n", "\n", " epochs = 100\n", "\n", " losses = list() # keep a list to plot later\n", "\n", "\n", " for _ in trange(epochs):\n", " values, params = update(params, xs, ys, optimiser_state)\n", " \n", " # append the loss\n", " losses.append(values)\n", " \n", " \n", " return losses, params" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "9cbe1682b83547c2af5b2689e24b69f5", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/100 [00:00]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAh8AAAGdCAYAAACyzRGfAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAK/dJREFUeJzt3Qm4VWWh//EvCgcDz3FCQVBxwilSLzjkFCqZU4pmSQ6hmV6nq3mjQq7ev6Eldus6hJp1TVLT0izMMcWuDc4jWuIQCqiAKEIeFASE9X/efd9z2ud4gDOsvdfee30/z/M+a+3N9uzXd7+c/WOtd+gGJEiSJJXJGuV6I0mSJMOHJEkqO698SJKksjJ8SJKksjJ8SJKksjJ8SJKksjJ8SJKksjJ8SJKksupOBerfvz8LFy7MuhqSJKkD6uvrmT17dvWFjxA8Zs2alXU1JElSJwwYMGC1AaTiwkfTFY9Qea9+SJJUPVc9wsWD9nx3V1z4aBIqb/iQJKn2OOBUkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVleFDkiSVVcVuLJe2/sDXgF7A2KwrI0lSjuXmykdf4ELgLOATWVdGkqQcy034eBaYAfQGPpd1ZSRJyrHchI/gt/H4hYzrIUlSnuUyfBwG9Mi4LpIk5VWuwsejwFvAesC+WVdGkqScylX4WAHcHs+99SJJUjZyFT6Kb70ckcf/eUmSKkDuvn//CCwA+gF7ZF0ZSZJyKHfhYxlwZzw/KuO6SJKUR7kLH4FTbiVJyk4uw8f9wAfAQGBI1pWRJClnchk+FgP3xHNnvUiSVF65DB+Bt14kSaqS8LHPPvtwxx13MGvWLJIkYcSIER97zbhx45g9ezaLFi1i8uTJbL311lSau4ElwPbAdllXRpKkHOlw+OjduzfPPfccZ555Zpt//u1vf5uzzz6b0047jd13350PPviA++67j549e1JJFgIPxHNnvUiSVF5JZ0swYsSIFs/Nnj07GT16dPPjhoaGZPHixcnIkSPb9TPr6+sLPzccu1K39pSvhv8HSKaU+H0stoF9wD5gH7AP1HofqO/A93eqYz622GILNt54Yx54oOmaAjQ2NvL444+zxx5tL+lVV1dHfX19i1Iut8d1P3YCti3bu0qSlG+pho9+/cK6oTB37twWz4fHTX/W2tixYwsBpamEsSTlElY6nRzPv1S2d5UkKd8yn+0yfvx4GhoamsuAAQPK+v6/jkfDhyRJVRg+3norbFgPffv2bfF8eNz0Z60tXbqUhQsXtijlFG69LAV2dNaLJEnVFz6mT5/OnDlzGD58ePNzYQxHmPXy6KOPUon+4a0XSZIqf6rtTjvtVChNg0zD+aabblp4fPnll3P++edz2GGHMXjwYG644YbCmh+33x6uMVQmb71IklReHZpKM2zYsKQtEydObH7NuHHjkjlz5hSm2E6ePDkZNGhQSabqpFXWhWRJnHa7fQVMV7LYBvYB+4B9wD5AlbVBR76/u8WTihFu04RZL2HwaTnHf9wFHApcAFxYtneVJKk2dOT7O/PZLpXi1nh01oskSaVl+Ih+F2e9DAZ2KHGjS5KUZ4aP6D3g/nju1Q9JkkrH8NHGrZejS9jgkiTlneGjyB3AknjbJdx+kSRJ6TN8tLr1cm88P6YEjS1JkgwfH3NzPBo+JEkqDa98tLHeR5idvAXw6RI1uiRJeWb4aGVxnHYbePVDkqT0GT5WcetlJLBmCRpdkqQ8M3y0YTIwD+gL7Ff+z0SSpJpm+GjDR0U73R5b3s9DkqSaZ/hYiV/G4xeAnuX7PCRJqnmGj5V4CHgDWAc4uLyfiSRJNc3wsRIJ8Kt47q0XSZLSY/hox62XzwP1KTa6JEl5ZvhYhWeBl4FPAEeU7zORJKmmGT7auebHcaX/LCRJygXDx2rcFI+fBfqV/vOQJKnmGT5W41Xg4bjSqQNPJUnqOsNHO9wYj6NSaHBJkvLO8NEOtwJLgJ2AT5X+M5EkqaYZPtphAXBXPP9KaT8PSZJqnuGjg7dewqwXG02SpM7ze7Sd7gHeBfoDw7vQ4JIk5Z3ho52WFS237q0XSZI6z/DRiVsvYafb3l1odEmS8szw0QGPA6/E4HFk6T4TSZJqmuGjk1c/vPUiSVLnGD466BdFy60P6GSjS5KUZ4aPDpoB/Ck2nFc/JEnqOMNHJ0yMx6925j+WJCnnDB+dcBvwPrANsFf6n4kkSTXN8NEJH8T9XgKvfkiS1DGGj066Lh5HuuaHJEkdYvjopIfjmh9rA1/s7A+RJCmHDB9d8PN49NaLJEntZ/joghuA5cAwYKuu/CBJknLE8NEFs4D74/mJ6XwekiTVPMNHSmt+nGBjSpLULoaPLroDmA9sGpdclyRJq2b46KIlwE3x/KSu/jBJknLA8JHimh9HAn3S+IGSJNUww0cKpgBPAnXAqDR+oCRJNczwkZKfxuO/pvUDJUmqUYaPlPwqbja3LfCZtH6oJEk1yPCRkhA8bo7np6T1QyVJqkGGjxT9TzyGvV7WT/MHS5JUQwwfKXoKeBZYC/hKmj9YkqQaYvgo0dUPb71IktQ2w0fKwoJji4BPAnum/cMlSaoBho+UNQK3xHOvfkiS9HGGjxKu+TESWLcUbyBJUhUzfJTAY8BfgU848FSSpNKHjzXWWIMLL7yQ1157jUWLFjFt2jTOP/988ubH8XhGxvWQJKkSJWmWsWPHJu+8805yyCGHJAMHDkyOOuqopLGxMTnrrLPa9d/X19cnQTimXbdylnpIGiFJINmvAupjsQ3sA/YB+4B9gFJ+73Xg+7t72klmzz335He/+x333HNP4fHMmTM55phj2G233ciThcCN8cpHKA9mXSFJkmr1tssjjzzC8OHDGTRoUOHxjjvuyN577829997b5uvr6uqor69vUWrt1ssRQP+M6yJJUiVJ9bJLt27dkvHjxyfLly9Pli5dWjiee+65K339BRdckLSl2m+7NJU/xVsvF1RAXSy2gX3APmAfsA9Qojbo4LCJdN985MiRyeuvv144Dh48ODn++OOTefPmJaNGjWrz9XV1dYWKNpX+/fvXVPgYGcPHrP+7x5V5fSy2gX3APmAfsA9Qa+EjBI8zzjijxXPnnXde8uKLL5ai8hVfekDyVgwgR1VAfSy2gX3APmAfsA9QgjboyPd36mM+evXqxYoVK1o8t3z58sIU3DxaVrTfy+kZ10WSpEqQeiK48847Oe+88zjkkEMYOHAgRxxxBN/4xjeYNGkSeV7xdDkwHNgu68pIklQBUr3ssvbaayeXXXZZMmPGjGTRokXJtGnTkosuuijp0aNH6pdtqqlMirderqiAulhsA/uAfcA+YB8g5TboyPd3t3hSMcJU28bGRhoaGli4MKyWURs+C0yOG88NAN7PukKSJGX0/Z3PgRgZeAB4EWgATsy6MpIkZcjwUUYT4vEsIFxykiQpjwwfZXQD8A9gG+Cgcr6xJEkVxPBRRh8AP4vnXy/nG0uSVEEMH2V2ZZx2e6DTbiVJOWX4KLMZwB1FYz8kScobw0cGfhSPJwDrZlEBSZIyZPjIwB+B54HewElZVECSpAwZPjK++vFvfgiSpJwxfGTkJuBdYAtgRFaVkCQpA4aPjHwI/CSefyOrSkiSlAHDR8Yrni4F9gZ2z7IikiSVkeEjQ2/F2y/B6CwrIklSGRk+Mvbf8fiFOP5DkqRaZ/jI2AvA74E1gXOyrowkSWVg+KgAP4zHrwHrZVwXSZJKzfBRAf4APBcXHTs168pIklRiho8Ku/pxNlCXcV0kSSolw0eFuAWYBWwMHJN1ZSRJKiHDR4VYBlwRz78JdMu4PpIklYrho4L8FGgEBgOHZF0ZSZJKxPBRQd4DronnYzOuiyRJpWL4qDCXAUuAveKy65Ik1RrDRwUuuf7zeO7VD0lSLTJ8VKAfAMvjuI8ds66MJEkpM3xUoFeBX8fzMRnXRZKktBk+KtQl8TgS2DLjukiSlCbDR4UKy63fGzecC+t+SJJUKwwfFWx8PH4V6JtxXSRJSovho4L9BXgEWAsYnXVlJElKieGjwn0vHk8HNsi4LpIkpcHwUeHuAZ4G1gb+PevKSJKUAsNHFbgoHs8C1su4LpIkdZXhowrcEWe/NABfz7oykiR1keGjCiRFVz++HkOIJEnVyvBRJX4LvACsG2+/SJJUrQwfVXT147vx/BtxAKokSdXI8FFFbgVeBtYHzsy6MpIkdZLho4qsKFr3Iyw61jvj+kiS1BmGjypzM/B3YEPHfkiSqpTho8osB8bF828580WSVIUMH1Xol8DUOPbDVU8lSdXG8FGlYz++E89D+HDVU0lSNTF8VKnb4qqn6wDfzLoykiR1gOGjitf9+H/x/GygT8b1kSSpvQwfVb7ny5NxwbExWVdGkqR2MnxUuaarH2HRsX4Z10WSpPYwfFS53wOPAJ8Azsu6MpIktYPhowY0hY5TgS0yroskSatj+KgBfwTuA3oAF2VdGUmSVsPwUSPOjcfjgJ0yroskSati+KgRU+LKp8H4jOsiSdKqGD5qyH8Cy4CDgWFZV0aSpJUwfNSQV4GfxvPvZ1wXSZLKGj769+/PjTfeyLx581i0aBHPP/88Q4cOLcVbqZUw4PQDYHfgCFtHkpSH8LHuuuvy8MMPs2zZMg4++GB22GEHRo8ezYIFC9J+K7VhLnBpPL8YWNNWkiRVmO5p/8AxY8bwxhtvcNJJJzU/N2PGjLTfRqvwQ+B0YHvga0W3YiRJqskrH4cffjhPPfUUt956K3PnzuWZZ57h5JNPXunr6+rqqK+vb1HUNY3AhfE8HG1RSVIlbpCaWlm8eHGhfO9730t23nnn5JRTTkkWLVqUjBo1qs3XX3DBBUlb6uvrU61X3koPSF6GJIHkuxVQH4ttYB+wD9gHarsP1NfXt/v7u1s8Sc2SJUsKVz722muv5ueuuOIKdt11V/bcc882r3z07Nmz+XG48jFr1iwaGhpYuHBhmlXLnRHA7cBiYBvgzawrJEmqWeH7u7GxsV3f36nfdpkzZw5Tp05t8dyLL77IZptt1ubrly5dWqhkcVE6fgf8KW46FwafSpJUCVIPH2Gmy7bbbtviuW222YaZM2em/VZqh9Hx+BVgiC0mSaoQqd7z2WWXXZKlS5cmY8eOTbbaaqvkmGOOSd5///3k2GOPTf2ekaV9bXBDHPvxoG1mn7EP2AfsA/YBsh/zkXr4COXQQw9Nnn/++cLA06lTpyYnn3xyqSpvaUcbbALJohhADrfN7DP2AfuAfcA+QI0NOC3ngBW133eB84C/A4PDWBsbT5JUKwNOVZkuAWYDg4Bzsq6MJCnXDB858T5wbjw/H+iXcX0kSfll+MiRXwCPxRVPw5UQSZKyYPjIkTC45+x4fgKwW8b1kSTlk+EjZ54EJsbzCUAYcSxJUjkZPnJobNx8Llz5GJV1ZSRJuWP4yKG5wEXxPIz9cNdbSVI5GT5y6grglTjrZVzWlZEk5YrhI6eWAWfF8zAIdceM6yNJyg/DR47dD/waWBO42sGnkqQyMXzk3L/HBcj2Ak7MujKSpFwwfOTcLOA78fy/gPUzro8kqfYZPlQYfPo3oA9wse0hSSoxw4f4CDg9tsMpwK62iSSphAwfKngIuD52iGviIFRJkkrB8KFm3wLmA0OAr9sukqQSMXyo2TvAN+P5hcDmto0kqQQMH2ohbDr3INAb+LFtI0kqAcOHPuZU4EPgIOBY20eSlDLDhz7m70Ubz13u2h+SpJQZPtSmH8S1PzYEfmgbSZJSZPjQSjeeC2t+rAC+Cgy3nSRJKTF8aKUeixvOBdcCa9tWkqQUGD60SucC0+O020tsK0lSCgwfWqUPgJPj+ZnAMNtLktRFhg+t1v8CP4nnPwN62WaSpC4wfKjdS6+/DmzlzreSpC4yfKhdFsbZL8FZwN62mySpkwwfarf7422XNeIy7N5+kSR1huFDHTIaeAPYOi5EJklSRxk+1CHvASfG8zOAA20/SVIHGT7UqdkvV8Tz69z7RZLUQYYPdcpY4CWgP3CVbShJ6gDDhzplMfAV4CPgy7FIktQehg912lPARfE87AEzwLaUJLWD4UNdcjHwBLAecIMdSpLUDoYPdUm47XI88D6wf1wJVZKkVTF8qMv+Hlc9Jd6G2dU2lSStguFDqfg5cAvQA7gZWNt2lSSthOFDqTkVmBlXP73SdpUkrYThQ6mufnocsBw4ATjGtpUktcHwoVQ9XDT99hpgK9tXktSK4UOp+y7wF6ABuBXoaRtLkooYPpS65fGWyzvAEOBS21iSVMTwoZKYFZdfb9r99mjbWZIUGT5UMvcB34vn/xNnwUiSZPhQSV0A/CmO//i14z8kSV75ULnGf7wN7AxMsMklKfe88qGSmxPX/1gBnAKcZJtLUq4ZPlQWDwD/Gc+vAoba7pKUW4YPlc144HfAWsBvgA1se0nKJcOHyiYBRsVdcAcCv7QDSlIuGT5UVo3AkcAHwAFFS7FLkvLD8KGyewH4Wjz/D+CLfgaSlCslDx9jxowhSRIuu+yyUr+VqsgtwH/H858DO2VcH0lSjYSPXXbZhVNPPZXnnnuulG+jKjUmroLaOw5E3TDrCkmSqjt89O7dm5tuuolTTjmFBQsWlOptVOULkH0ZeCUOQA0zYHpkXSlJUvWGj6uuuoq7776bP/zhD6t8XV1dHfX19S2K8uMfwOHAe8A+wJVZV0iSVJ3hY+TIkQwZMoSxY8eu9rXhNY2Njc1l1qywH6ry5OW4BHtYAfVfgX/LukKSpOoKH5tssglXXHEFxx13HEuWLFnt68ePH09DQ0NzGTBgQNpVUhW4N44BCS4HDs64PpKk0ukW135KzYgRI7j99tv56KOPmp/r3r07K1asKJSePXsWjisTbruEKyAhiCxcuDDNqqkK/Czu/RI++b2Av2ZdIUlSu3Tk+7s7KQtjPAYPHtziuYkTJ/LSSy/x/e9/f5XBQzoN2BzYH7gL2B14y2aRpJqSevh4//33eeGFsIzUP33wwQe8++67H3team0ZcBTwKLAdcCcwDFhkU0lSzXCFU1XkDJhDgXlhrRjgRjuqJNWU1Md8dJVjPtQkjPkIE7V7AlcA59g0klSxOvL97ZUPVayHgRPi+deB0RnXR5KUDsOHKn4PmG/E8x/G9UAkSdXN8KGKF7YkvLRoE7owE0aSVL0MH6oK34xXQeqASe6CK0lVzfChqhBGRY8CHgQagN8DW2VdKUlSpxg+VDWWAkcCU4B+wGSgf9aVkiR1mOFDVSXsfnsg8HdgC+B+YP2sKyVJ6hDDh6rO28ABQNj/+JPAPUDvrCslSWo3w4eq0swYQN6N+7/cHhcjkyRVPsOHqtaLwMFxB9zPAr8GemRdKUnSahk+VNWeBA4HFgOHAb8E1sy6UpKkVTJ8qOr9ETgCWBJ3xHUjOkmqbIYP1YQw6+WLwLK4BPvP4q6JkqTKY/hQzbgL+DLwEXAicI0BRJIqkuFDNeW3wFeA5cC/Aj8xgEhSxTF8qOb8Ki7FHgLIKcC1dnRJqiiGD9Wkm4HjYwA5CbjOzi5JFcPwoZq+AnJMHANyAnC903AlqSIYPlTTwsJjI+MsmOPjFREXIpOkbBk+lItBqF+Ku+IeDUwC1sq6UpKUY4YP5cLv4gqoi4BD42Z0a2ddKUnKKcOHcrUQ2YFAI7Af8ACwXtaVkqQcMnwoVx4C9gfmxd1ww9LsG2ddKUnKGcOHcudpYBgwG9gReBgYlHWlJClHDB/KpanAXsDfgS1iANkl60pJUk4YPpRbM2IAeQrYEHgQOCDrSklSDhg+lGvvxMGnk+Psl7uB47KulCTVOMOHcu/9OP32l3EBsl8A5+W+VSSpdAwfUlwBNVzx+K/YGt+NG9J1t3UkKXWGDylKgDHAGXFDuq/F2zANtpAkpcrwIbXyY+BwKNyO+VxcG2SgrSRJqTF8SG24J64FMgf4FPAEsKctJUmpMHxIK/EMsBvwLLAR8L/AKFtLkrrM8CGtwpvA3sBvgJ7A9cAl/sWRpC4xfEirEXbC/VKcAUMclBp2yV3HlpOkTjF8SO2cCfOfwLHAYuDzcRzI9raeJHWY4UPqgF/G2zCvA9sAjwNH2IKS1CGGD6kTA1GHxr1g6oFJwIX+ZZKkdjN8SJ0wL25Cd2l8HG7J3Av0sTUlabUMH1InhVVQR8dl2T+IC5KFabl72KKStEqGD6mLbo7rgbwEbAL8CTjHVpWklTJ8SCmYCuwK/CrujHtZXBtkXVtXkj7G8CGlJOwFcwxwJrAU+AIwxWXZJeljDB9Syq6O4z6mxQ3pwm2Ysf5lk6Rmhg+pRNNxhwA3Ad2Bi4H7gI1tbUkyfEilshA4Hjgxzob5LPDXeDtGkvLMKx9SiV0fFyV7GtggDkS9Li5QJkl5ZPiQyuDlOA4k3H5ZAXzVwaiScszwIZXJMuA8YBgwA9gS+AvwX0BPPwVJOWL4kMrsIWAn4OfxL+C34sqoYZ0QScoDw4eUgcZ46+UwYA6wPfAo8D2gzk9EUo0zfEgZugv4ZJySuybwH+4PIykHDB9SxhbEKblhCu5bwA7x1swVQO+sKydJJWD4kCrEpBg8Jsa/mGcDLwAHZl0xSar08HHuuefyxBNP0NjYyNy5c5k0aRLbbLNN2m8j1exVkJOAzwHT4/Lsvwd+CfTLunKSVKnhY9iwYVx11VV8+tOf5oADDqBHjx7cf//99OrVK+23kmrWZOBTwKXAcuDLwEvAGV6ulFQjklKWPn36JME+++zTrtfX19cXXh+Opa6bxTaohj6wMySPQ5LEEs6HVEC9LLaBfcA+QCe/v0s+5mOdddYpHOfPn9/mn9fV1VFfX9+iSPqnKXF11HDV4z1gN+BJ4Jq4XLskVZuSho9u3bpx+eWX89BDD/HCC2Ho3MeNHTu2MD6kqcyaNauUVZKqUliS/cfAdsCN8S/uqcArwOneipFUhUp26ezqq69Opk+fngwYMGClr6mrqytcomkq/fv397aLl/K8nLuaPrA3JFOKbsU8C8kw+439xj5gHyC7NujgsInSVGLChAnJ66+/nmy++ealrLzFNshtH1gTkjMgmV8UQn4DyZYVUDeLbWAfyF8fqM96zMeECRM48sgj2X///ZkxI2yhJSltYRbM1cAg4Ergo7hQ2Ytxs7r/G20lSZUn9fARptkef/zxHHvssSxcuJC+ffsWylprrZX2W0kC3gXOipvV3Rf3hgmb1U0Dvu5eMZIqVKqXXVbmhBNOSP2yjcU2sA98vA8cDMkLRbdiXoXky5B0s7/YX+wD9gFq9LZLmOHSVrn++uvTfitJbbgX2BE4BZgNbBlXSH0yrpwqSVlzbxepRseDXBvHg5wHNAJD422ZPwJ7Zl1BSblm+JBq2CLgYmAr4L+BD8MWCMDDwF3Av2RdQUm5ZPiQcmAe8E1ga+AncWbMocAzcTfdnbOuoKRcMXxIORLWDz4trpT6i3h75gjgWUOIpDIyfEg59CrwFeCTbYSQO4FPZ11BSTXN8CHl2MtFIeSmGEI+DzwK/C8wPOsKSqpJhg9JhRByfLwdE2bJLAX2Ax4AHgeO8peFpBQZPiQ1mxbXBwmzY34ELAZ2A26LV0nCTrquVSypqwwfkj7mzbg0+2bAuLiEe5gpcw0wE/gOsJHtJqmTDB+SVjlF9zsxhIT9Y2bE0HEB8DpwXVxNVZI6wvAhqV2LlV0Zr34cHQek9gS+CjwXB6ceCaxpW0pqB8OHpHYLs2F+HZdnD9Nxb4kLloXBqb8FXgPOBfrYppJWwfAhqVPCLJgvx43rwhLu78TbM+OBN4Ab3ENG0koYPiR1yRtx87pNgVFx99y14vohYQ+ZvwJnAuvYzpIiw4ekVCwBboxTc3eN64WEsSKD43iR2cDPgX1sbyn3DB+SUvdUXC+kP/Bv8epHL+AE4M/AS8C3gX62vZRLhg9JJfMecFWcjhsGqP4PsBDYFvh+XE/kHmCki5dJuWL4kFS2Aar/CmwMnBTHg4SpuQcDvwLmxEXMwm2Zbn4mUk0zfEgqqw+AicDewCDgorhq6rpx+fZwW2Z6nEGzg5+NVJMMH5Iy3Uvm/wFbxLVCrou3agYCY4EX4iJmY+OUXkm1wfAhKXMJ8Efga3EQ6peA2+PuujvGqyCvAk8Ao2M4kVS9DB+SKsqHcRfdI2MQCYHk/ri6apjC+8O4x8wTccaMV0Sk6mP4kFSxFsRbMQfGgaqnx31kmoLI9+MVkWfjZnc7ZV1hSe1i+JBUFd6Js2GGx/VDwuDUyXFvmZ3j7rtT4v4ylwL7At2zrrSkNhk+JFWdt4GfAp8D+sbFy8IYkcVx8Oq/Aw/GwHIzcAywXtaVltSsWxzrVTHq6+tpbGykoaGBhQvDckSS1D5hFdUDgCOAQ4ENi/4s3Kp5NC5qdk+cRSMpm+9vw4ekmr2suztwWCxhj5lis+JA1vvi7Zv5GdVTqhWGD0lqZbO4mmoonwV6F/3ZirgfTQgjD8QrJGGar6T2M3xI0ir0jMu4fy7OpAlriRRbFFdafSDOrnkuBhRJK2f4kKQO2DgGkc/G0q+NKb9/ioNYQ/lbpQ2WkyqA4UOSuuCTMYSEab2fAdZp9edhfMhf4tWRP8d1RsKAVinP6h1wKknpCDvvDol7z+wXN8Rbu9Vr3gceAx6K5bG4gZ6UJ/WGD0kqbRj5TCz7tLGGSLgK8nwcuPpIPIbFz6RaVm/4kKTyCOsV7BCviDSVzVeyMFrYj+bxWJ6IO/hKtcLwIUkZCsu/7wHsGY9D4gyb1l4BnozlqTh2JMy0kaqR4UOSKkjPuP/MbnHhs1C2buN14XbNS8AzwNPxGParca1nVQPDhyRVuPWBXeLuvE0lXDFpy7QYQppKWHfkzTLXV0ozfLjpoyRlYH5cUTWUJv3iLZqmMjSuzLp1LF9s9d8/H8tf49ojf4szb6RKZ/iQpArxVtHGd8VXSHYC/iWWsBrr9vH5fWMpNiOGkBdimQq86FgSVRjDhyRVsPlFK6s2qQO2i6EkhJFPxY3zBsSZNqF8vo1Q8mIcU/JiLC8D75T5/0cKDB+SVGWWFt1yKbZ+DCGfjNN/m459i0JJ2Fivdbh5uaj8Pc7CCeNMFpfx/0n5YviQpBoxv2jJ92IbxFs128Vj0/nAGFj2iKW1N2IIKS6vxgXTnIGjrjB8SFKNe7do6fdiawGDgG1j2SY+3iYGlk1jCcvKt/ZODCEhjEwvKq/F0OJeN1oVw4ck5dSHcaZMKK2tH4PIVvG4dTxuCWxYVMKaJa19BMyK40xCmRnL6/H4Rnxv5ZfhQ5LU5i2cpqXgW6uPIWTLGE62KCqbxysqA2MZtpK2fTuGkKbyely7pKnMjmNbVJsMH5KkDlkYFzoLpa29booHuDaVzWIZGHcF3iiWsJYJqwgos1qV2UVlTrz9k/j5VR3DhyQpNUlcrySUx1bymnVjCNm0VdmkqKxVFFDC+iYrE27xzI1BZE7Re78Vn286huIg2cph+JAkldU/YmnrykmTDWIIGRBL/6JjKBvHYNK96DWrszheTWkKI+/Ex2/H89bFcSmlY/iQJFXkDJ13VxNQusdbPP1iGCk+Nj3fVHoDnygai9IeYan6ea3Kuysp82Px6kr7GD4kSVWpaVZNKKvTKwaSjYqOGxbd2ml6HEqfuIrs2rGEMSvttQxYEINI8bGt8o9WZWGOxq8YPiRJNW9R0Vok7dEQg8gGMYxsUPR4g1bPrx+PYZxKj6Iw01ErgMYYRN5rVRpbnbdVFsZjNdwuMnxIktRK0xd6WEStvdaKIWS9GEiajuvHQbbrFZV1Wz3XE1ij6PmuXhFaWBRGFsZbSMXHMJ35kgw/dcOHJEkp+LADt4HaCi7rFJV1Wz1uKg2tyjpx3ZWGeGz6Ym8KNSsTNhg0fEiSlPPg8mGchdNZ3eIYlfpWgaS+6PmmY7i1Qy1e+TjjjDP41re+Rb9+/Xjuuec466yzePLJJ0v1dpIk5VpSdLul0oVbTKk7+uijufTSSxk3bhxDhgwphI/77ruPDTcMw3UkSVLeJWmXxx57LJkwYULz427duiVvvvlmMmbMmNX+t/X19UkQjqWom8U2sA/YB+wD9gH7AKm3QUe+v1O/8tGjRw+GDh3KAw888M90kySFx3vsscfHXl9XV0d9fX2LIkmSalfq4aNPnz50796duXNbDpsJj8P4j9bGjh1LY2Njc5k1qzPjhCVJUrUoyZiPjhg/fjwNDQ3NZcCA9qzQL0mSqlXqs13mzZvHRx99RN++YQHbfwqP33or7C/Y0tKlSwtFkiTlQ+pXPpYtW8bTTz/N8OHDm5/r1q1b4fGjjz6a9ttJkqQqU5J1PsI02+uvv56nnnqKJ554gnPOOYfevXszceLEUrydJEnKe/i49dZbC2t6XHjhhYVBplOmTOGggw7i7bffLsXbSZKkKtKt0nbwDVNtw6yXMPh04cJqWKdNkiTVd+D7O/PZLpIkKV8MH5IkyfAhSZJqV8l2te0ql1mXJKl6dOR7u3ulVt5l1iVJqj7he3x1A04rbrZL0L9//5LMdAkNEkJNWMLdmTSlZVuXj21tW9ci+3V1tnX4WbNnz66+Kx9BeyreFaFxDR/lYVuXj21tW9ci+3V1tXV7/3tnu0iSpLIyfEiSpLLKVfhYsmQJ3/nOdwpH2da1wn5tW9ci+3Vtt3VFDjiVJEm1K1dXPiRJUvYMH5IkqawMH5IkqawMH5IkqaxyEz7OOOMMpk+fzuLFi3nsscfYdddds65S1Tv33HN54oknaGxsZO7cuUyaNIltttmmxWt69uzJlVdeybx58wqLz9x2221stNFGmdW5VowZM4YkSbjsssuan7Ot011l+cYbbyz020WLFvH8888zdOjQFq8ZN25cYUHE8OeTJ09m6623TrEG+bDGGmtw4YUX8tprrxXacdq0aZx//vkfe51t3Tn77LMPd9xxR2H10vD7YsSIER1u2/XWW49f/OIXvPfeeyxYsIBrr72W3r17k4ak1svRRx+dfPjhh8mJJ56YbL/99slPfvKTZP78+cmGG26Yed2qudx7773JCSeckOywww7JjjvumNx1113JjBkzkl69ejW/5uqrr05mzpyZ7LfffsmQIUOSRx55JHnooYcyr3s1l1122SV57bXXkilTpiSXXXaZbZ1y+6677rrJ9OnTk+uuuy7Zddddk8033zw54IADki233LL5Nd/+9reTBQsWJIcffnjyqU99Krn99tuTV199NenZs2fm/aOaytixY5N33nknOeSQQ5KBAwcmRx11VNLY2JicddZZtjVdb9+DDjooueiii5IjjjgiCUaMGNHiz9vTj++5557k2WefTXbbbbdkr732Sl555ZXkpptuSuPzz74Dlro89thjyYQJE5ofd+vWLXnzzTeTMWPGZF63Wip9+vQpdPB99tmn8LihoSFZsmRJ4RdK02u23Xbbwmt23333zOtbjaV3797Jyy+/nAwfPjx58MEHm8OHbZ1eG48fPz7585//vMrXzJ49Oxk9enTz49D+ixcvTkaOHJl5H6mmcueddybXXntti+duu+225MYbb7StSbet2wofq+vH2223XeG/Gzp0aPNrDjzwwGT58uXJxhtv3KX61Pxtlx49ehQulz7wwAPNz4XLT+HxHnvskWndas0666xTOM6fP79wDO1eV1fXou1ffvllZs6cadt30lVXXcXdd9/NH/7whxbP29bpOfzww3nqqae49dZbC7cTn3nmGU4++eTmP99iiy3YeOONW/TrcOvx8ccft1930COPPMLw4cMZNGhQ4fGOO+7I3nvvzb333mtbl1h7+nE4hlstTz/9dPNrwutXrFjB7rvv3qX3r8iN5dLUp08funfvXvglUiw83m677TKrV63p1q0bl19+OQ899BAvvPBC4bl+/foVVswL9wpbt334M3XMyJEjGTJkSJvjlWzr9Gy55ZacfvrpXHrppVx88cWF9v7Rj37E0qVLueGGG5r7blu/U+zXHXPJJZfQ0NDASy+9xPLly1lzzTU577zzuPnmmwt/bluXTnvaNhzffvvtFn8ePqfwD8yu9vWaDx8q37/IBw8eXPhXi9K3ySabcMUVV3DAAQe4PUAZBkGGKx/hSzCYMmVKoW+fdtpphfCh9Bx99NEcd9xxHHvssYV/tOy8886Ff8SEAZC2dW2r+dsuYbT6Rx99RN++fVs8Hx6/9dZbmdWrlkyYMIHPf/7z7LfffoVR1U1C+4YZGE23Y5rY9h0XbquEdgu3AJYtW1Yo++67L2effXbhPPxrxbZOx5w5c5g6dWqL51588UU222yzwnnT7w1/p3TdD37wg8LVj1tuuYW//e1vhVkVYQbX2LFjbesSa08/DsfWsxPD1an111+/y9+fNR8+wi/mcL8q3FcsvkUQHj/66KOZ1q1WgseRRx7J/vvvz4wZM1r8WWj3cKm6uO3DVNyBAwfa9h0UxniEf32Hfxk2lSeffJKbbrqpcB7+pW5bp+Phhx9m2223bfFc6LdhrFIQpuyHgFLcr+vr6wv3wP2d0jG9evUqjB9ofVk/XH2yrUurPf04HMNU23C7t0n4XR8+nzA2pKuSPEy1DSN4R40aVRi9e8011xSm2m600UaZ162ay1VXXVWYpvWZz3wm6du3b3NZa621Wky1DdNv991338JU24cffrhQsq57LZTi2S62dbpTmZcuXVqYBrrVVlslxxxzTPL+++8nxx57bIspiuF3yGGHHZYMHjw4mTRpklNtO9HWEydOTN54443mqbZhSujbb7+dXHLJJbY16cyO22mnnQolOOeccwrnm266abv7cZhq+/TTTxemne+5556F2XZOte3Ah3DmmWcWvgTDeh9h6m2Ys5z1l0e1l5UJa380vSZ04iuvvDJ59913C7/Af/Ob3xQCStZ1r8XwYVun17aHHnpo8vzzzxf+0TJ16tTk5JNP/thrxo0bl8yZM6fwmsmTJyeDBg3KvE9UW1l77bULfTj8bl60aFEybdq0wroUPXr0sK3pevsOGzaszd/RIfS1tx+vt956hbAR1l/5xz/+kfzsZz8rhJqu1q1bPJEkSSqLmh/zIUmSKovhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IklZXhQ5IkUU7/H9IXjxwlx636AAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "\n", "\n", "plt.plot(jnp.arange(100), losses, c=\"r\")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAGdCAYAAAA8F1jjAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAOtlJREFUeJzt3Qd4VFX+xvGXFhAIHWkCIqCoKIhKs2BBZd1dcV0F1N1FsevfigVwXYKgWAFFRQQF0bXAKooVRbADYkNFBaVJL9J7cf7PubkT7pxMQiaZmXtn5vt5ntHcMyUnd0Ly5txzfqeUpJAAAAACqLTfHQAAACgIQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAAQWQQUAAARWWaWB+vXra/PmzX53AwAAxCA7O1vLly9P76BiQsqyZcv87gYAACiGBg0aFBpWUj6ohEdSzBfKqAoAAKkzmmIGGvb3uzvlg0qY+UIJKgAApBcm0wIAgMAiqAAAgMAiqAAAgMBKmzkqhSlVqpSqVavmTNwxHyO1hEIhZ/7Rhg0bnI8BAJkj7YNK7dq1dcUVV6hFixZ+dwUl9PPPP2vUqFFas2YN5xIAMoQZXkjpP1HNKMmmTZtUpUqVfKt+ypYtqyeeeEJbtmzR+PHjtXr1au3du9e3vqJ4ypQpowMPPFDdunVT5cqVde2112rPnj2cTgBI09/fGTOiUq9ePVWoUEEPPfSQ5s2b53d3UAILFizQunXr9O9//1t169bV0qVLOZ8AkAHSejJt6dK5X97OnTv97griIPw+mhEWAEBmSOugAgAAUhtBBQAABBZBBTEZM2aMJk6cyFkDACQFQSUN9e/fX998843f3QAAoMQIKgAAIKo/S7pdUlX5J+OCSkWfbrE666yz9Mknn2j9+vVau3at3njjDR1yyCF59zdo0EAvvPCCfv/9d6dOzKxZs9S2bVv17NlTOTk5at26tVPF1dxMW+PGjZ2PW7VqlfcaVatWddo6deqUt0pq9OjRzlLgbdu2OQXWbrjhhricdwBAapku6U1J90s6x8d+pHUdFZsJDFt9+tyVJG2L5fGVKmnIkCH67rvvnCJnd999tzM3xASQihUr6qOPPtKyZct0zjnnaOXKlWrTpo0TNF5++WW1bNlSXbp0UefOnZ3X2rhxo+rUqbPfz2meb+qTXHDBBU4A6tixo5566imtWLFCEyZMKMFXDwBIFfUkLbfa/JyZmFFBJZW8+uqrEce9evVyRlaOOOIIJ0CYrQGOP/54Z8TFmD9/ft5jzQiLqdy6atWqmD6neY4ZjQlbtGiROnTo4FSEJagAQPrrKWms5/h3SQdK+sPHPmVUUNnmjmz49blj0axZM2cUpV27dqpVq1Ze8bpGjRo5oypmsmw4pMSTKU9vQpH5PAcccICysrL07bffxv3zAACCZbakoz3Ht0l6SP7LqKBSnMDgFzMnZfHixc6GisuXL3eCypw5c5zgsH379phf748/cvOwd/focuXKRTyme/fuznYDvXv31vTp0529F2677TYnLAEA0lNDSb9ZbYdK+kXBkHGTaVNBjRo1nN2eBw0apKlTpzqTWqtXr553v5m3YkZVvG1eu3btyldmPrzjsNn/KMy8htcJJ5ygzz//XCNGjHBGUczlpKZNm8b5qwMABMWVVkhZ6gaDoIQUg6ASQOGVPldeeaUTFE499VRnYm3Yiy++6Eygfe2115z5Kk2aNNF5552n9u3b580tMW1mhU/NmjWdUZgdO3Y4oyR9+vRxQtDJJ5/sBCGvX375Rccdd5zOPPNMNW/e3Ln0ZObBAADSSyk3jIz0tN3gjq6EFCwElQAyS4Z79OihY489Vj/88IOGDh3qXIIJ2717txMmVq9erbffflvff/+9E0D27t3r3P/KK6/o3Xff1bRp05zAc+GFFzrtZu5J2bJl9dVXX2nYsGHOTsReI0eOdCbxmpVDM2fOdELOE088keSvHgCQSE3cybHNPG2m+MXwAJ/2UCrfsrOzQ4b5v31f48aNQ+PGjXP+73c/uZX8HPB+8n3EvyO+B/geUInOwfXmb2HPba4UKhXA39/eW8ZNpgUAIBMv9SwxxUKt+SmjFHwEFQAA0lhzSfOstobuxNlUwBwVAADS1G1WSPnWHV1JlZBiMKICAECaKS1praTqVtXZcUo9BBUAANLIEZLmWG31Ja1QauLSDwAAaeIuK6R87l7qSdWQYjCiAgBAiisraaukLE9bD0kvK/URVAAASGFHuxsKepkdj3M3TimhgyXVMnu3mP1Z5Asu/UALFy7UjTfeGFEZt2vXrkk/M/3793d2hQYAFM09Vkj5wL3UU+KQYl7kGkmXSPqLpMP9e0cYUUE+devWdfYbKmq4OPfcc3XMMcdwJgEgScpFGeA4T9LEeLy4qQp3hdX2s3xDUEkT5cqVc/YAiodVq1bF5XUAAPF3rKQvrbaaktbF48XNxJYWnuN17iZAppi9T7j0E1BmQ8Hhw4c7tw0bNmjNmjXObsbeyzVmU8Fnn31WGzdu1FNPPeW0n3DCCfr444+1bds2/fbbb3rkkUdUsWLFvOfVrl1bkyZNcu5fsGCBLrroonyf277006BBA73wwgv6/ffftWXLFs2aNUtt27ZVz549lZOTo9atWzvPMTfTZlStWlWjRo1yNk40/fvggw909NHmSuo+d9xxh7ML9KZNmzR69GhVqFAhIecSANLFECukvOlepSlxSKkhKccKKS9JetTfkJKZQaWcT7diML/09+zZ44QCM4fklltu0eWXX553/6233qrZs2c7l10GDhyoQw45xNk12eyebEJB9+7ddeKJJ+qxxx7Le87YsWPVsGFDnXrqqTr//PN17bXX6sADzbSr6CpVqqSPPvrICSvnnHOOWrVqpQceeEClS5d2dll+6KGHnB2ezeUiczNtxoQJE5zX/dOf/uTsAv311187YaV69dzyQxdccIETcvr166fjjjtOK1ascPoCAMivvJsXbva0/VnSX+NxsrpIusFzbD7RIH8v92TupR8TGO70ccZTjFdmlixZoptvzv22nDdvno466ijn2Iw+GFOnTtWQISZf5zIjGP/973+dURTj119/1Q033OAEjWuuuUaNGjXS2WefreOPP15ffpmbyS+77DL9/HPB341mxMWMwpjnhOetzJ8/P+9+M8JiwpT3cpEZ1THhygSVXbtyr6LedtttzlwWE45MP2+66SY9/fTTeuaZZ5z777rrLnXu3JlRFQCwdHDroXhVk7SxpGeqkltjX9YQjX1dyWeZN6KSQmbMmBFxPH36dDVv3twZzTDCYSPMjHZccskl2rx5c95t8uTJKlOmjJo0aaLDDz/cmcfy1Vdf5T1n7ty5hU6cNZd1zEqcok6uDfejcuXKzqUib19MH5o2beo8xvRl5syZ+b4+AMA+T1ghZYJ7qafEIaVjlJByf/BCSuaNqOx2Rzb8+txxtnWrKe+zjwkHI0eO1KOPmouKkcx8lUMPPTTmz7F9+/aYn2P6YS7lnHLKKfnuM/NtAACFMzMLI3/CS2dImlLSE2cqwvWz2j42Q/TBfUcyK6gkKDAkSrt27SKO27dvr19++UV//PFH1MebeSBHHHFExKUZL3OJx6wOMnNGwqMxJryE541E89133znzYsxjoo2qmEs7ZsTG7oeZr2IuCS1evDjq6/7000/O1/fcc89FfH0AkOlOlvSR1ZZtLrWX9IWPkvR3q21oPIZnEotLPwFm5pQ8/PDDTpjo0aOHrr/++rz5J9Hcf//96tixo7NSyFx+adasmTMB1hyH57m88847zqiLmUPSpk0bZ76LWQFUkBdffNFZmfPaa685r20u35x33nl5oWLRokVOm/l8NWvWVFZWlqZMmeJcxjHPOeOMM9S4cWN16NBBgwYNckKSYb6OXr16OZeqzOUsM7H2yCOPjPs5BIBUMsYKKc+6l3q2lPQ3/W1WSJntrvIJeEhJeFA56aSTnKWwy5Yti1rtdMyYMXnLWsM384sUucaNG6cDDjhAX3zxhR5//HHnl3t4GXI033//vTp16uQEm08++cSZW2KWNC9fvjzvMZdeeqlzbCbYvvrqq87rmSXEBTFzWs4880znMW+//bbzOfr06aO9e/c695sVRmalkVlOvXbtWl144YVOu5m0a5ZJm/fYBKSXXnrJCSzhSbfjx493ViqZFURmzoy5b8SIEbz1ADJSZXexjSkEG3aKdVwsTST9x504GzYiXpXhkieUqFuXLl1CAwcODJ177rkho2vXrhH3jxkzJvT222+H6tSpk3erVq1aTJ8jOzvbeW3zf/u+xo0bh8aNG+f8P5FfZyJu06ZNCw0dOtT3fgTplsrvJzfOAd8DfA8U9D3Q2ZSvsm4V43G+LldIOZ5br2C9B4X9/vbeEjpHxfylbW6F2blzJ5VQAQAZydRU6+45ftLdYqdE6kq62moba67VKyX5PpnWrAwxlwPMRE1TF8RUW123ruAae2YORPnypvRNruxsM8UIAIDUUdWsgoyyYrjERRrOl9TSc7zZnTAbfQ1GSvA1qJjRFjNPwpSDN/U17r33XmeOipl4WdDKlr59+zoTL9OdqRwLAEg/Z0t6y2o7QNKOkrxoNUk3WW2m6MocpTxfg0q43LphyrCbpbBm/xkzymJGV6IZPHhwRDVWM6JiJusCABB0k6yy98OssvjF0lnSiSWvhh5Uvl/68TIjK2bzPbOstqCgYup2hMuyAwCQCsyef79bbceXtBBsRUm3W21mWmhkUfOUF6igYja+M7U4TFXTeDDLnY2yZQP1ZaKYwu9j+H0FgFRwbpTVwGamZYn+5G7rXkPyejBKOds0kNDf4GbnXTM6EhYuDGYmy5pb//79nTocpqCYmaNiamqYjfTM/jTxYPaaMVq0aFFgtVakDvM+GqZeCwCkAlPy/nTP8eAoFexLvLnu55LeU9pKaFA57rjj9OGHH+YdDx1qph5LY8eOdXbzPfroo9WzZ09Vq1bNKUL23nvvObvoxuvSjtkLx3z+bt265ZWQN2XdkXojKSakmPfRvJ+FVdIFgCCoLckupdnaLQhbbIdba5kNU6y86HvGpiRTmTelx9HNZNpNmzapSpUqzg69tlKlSjnVWKNtkIfUYkJKuJoxAARVd7c+SthOt/LsnpLUkL/RXdMc9qMp8S2l8+/vjAkqYRUrVlStWrWc4ILUYoKJudzDSAqAoPvMrYcS1l/S3SV5wcZm7xOrbaSk+EzlTInf3xkzy9T8kvvtt9/87gYAIA3VjZIdjnQHPoqtp7tXT5j5BE+l+vBC7DImqAAAkAj/NJvIeo5NxdlaknK3bi3mBJfrrLbnJGXomhCCCgAAxfStpFae4zskPVCSs9lV0jGe4x3usuNip57UR1ABACBGB0laYrUdKumX4p7JKpJusdomlnSZUHogqAAAEIPLJY3yHC+X1LAk+/6ZRan2wtTB7nIhEFQAACique7ISZhZNfxocU9fBUl9olSI+5T3w4sRFQAA9uNgsx+d1XZIlLYiayPpHKvtIUlbeCtsBBUAAArxf5KGe47NPJTDirtKuKw7iuL97TtL0lu8BQUhqAAAEIUpD7rYnX8SdrVbb61YzDWji6y24VG2VUYEggoAAJZmUVbwNIqy0qfIiec6t7hK2DxJL3Dai4KgAgCAR293ukjYd1atlJjXMZtlQl6jJS3llBcVQQUAAHfvvzWSanjOxiWSni3u2blYUnPPsXnxJzKvBH5JEVQAABnv8Cj78tQv7t5/NSVdb7W94F7uQbECJAAAGetOK6TMcKeVFCuknG2FFFP6fhAhpSQYUQEAZOwvwM1u3bWwCyW9VJwXqyzpVqttkqSvS9ZHEFQAABnoKHeSrFcdSauL82InSDrDarvP3VAQJcaICgAgowxyL/eETZN0WnFeqLykvlbbh+4NcUNQAQBkhHKSdlltf5f0anFe7GhJ51ltQyRtKn7/EB1BBQCQ9o6V9GWUxTnrYn2hMpJusya2fCPp9ZL3EdGx6gcAkNYeskLKW+6qnphDSlNJd1khxdRFIaQkFCMqAIC0VD7KfNa/FGf/P5NqrpRUz9O2sCSV4BALggoAIO20c+uheFWXtCHWFzLh5CqrbYy7WyGSgks/AIC08rgVUia4gyIxh5RuVkgxL3A3ISXZGFEBAKSFAyRts9rOlPR+rC9khl5utNrGR6mxj6QgqAAAUt5Jkj622qq4lWdjYpJNR6vtHkm7S9Y/FB9BBQCQ0p6RdKnn+DlJ/4r1RSpKut1qe1vSFyXvH0qGoAIASEmVJG2x2k4tTmHY9pK6WG0PRLmOBF8QVAAAKed0SVOiBJdtsZaq9dbSNz6N8sLwFat+AAAp5UUrSzzlruqJKaQcGSWkDCOkBBEjKgCAlGAmx26MsnHx57H+eX6zpGxP2w+S/hefPiL+CCoAgMD7kzu31V6ObFeeLdTBki6x2p6UtLLk/UPiEFQAAIH2mqSunuNHJN0U64v0ktTIc7xU0uj49A+JRVABAARS9SgbB7aVNCuWF6kj6RqrbZykBSXvH5KDoAIACJyu7kiKvcngrlhe5G+SWnmOt0p6WNIf8ekjkoOgAgAIlPckneE5vk9S31heoKo7YdbrFUnfx6d/SC6CCgAgEGpJWmO1HSPp21he5DRJJ1tt98Y6FIMgIagAAHx3gbvvX9hut6r9nqK+gFkCdEeUoZmY1i4jiCj4BgDw1adWSMmRlBVLSDkuSkh5kJCSLhhRAQD4oq6kFVZbS0lzYvkNdqdbljZshqR349dH+I+gAgBIun+4uxyHbZJUQ9Leor5AC0k9rLZHo6xnRsojqAAAkuorSW08x30k3V/UJ5vRk+vdVBP2s6SX4ttHBAdBBQCQFA3cgrBeh0maV9QXaCjpMqttlKRl8ekfgomgAgBIuMvdTBFm5qYcFEvttX9Kauo5XuXu0xOKbz8RPAQVAEBC/eyOnISZWmzDYimu8n9W238l/RK//iHYCCoAgIRoLGmR1dY0lm12/uIuPQ7b5U5mKfKMW6QDggoAIO6uk/SY53i+pOZFvVKTLam31fZarCVqkS4IKgCAuDGLcha7817DrnGnkxTJyW4ZfK/BknbGr49ILQQVAEBcmMs6v0a5/PNbUZ5cPsrOg1Mlfcybk+kooQ8AKLFbrJDygzu6UqSQ0jpKSHmYkIJcjKgAAEr01+4qd3FOWC9JY4ry5DLuHj1mYx9vNbg3eEOwD0EFAFAspor9T1GKui0vypPNzNqLrbbHJa3hzUAkggoAIGb9JN3jOf5CUruiPNFcD7paUh1rSZB34x8gWXNUTjrpJE2aNEnLli1TKBRS165d8z1mwIABWr58ubZt26b3339fzZo1S2SXAAAlYK7WbLVCysVFDSlmuKW/FVKeJqTAx6BSqVIlzZ49W9ddZ1bU53f77bfrhhtu0NVXX6127dpp69atmjx5ssqXN9O/AQBBcpSkPZIqetpM5nihKE82Ox1f4Tk2uxwPkLQk/v1E+gkl42Z07do1om358uWh3r175x1XqVIltH379lD37t2L/LrZ2dnOa5v/J+tr4cY54HuA74FM+x642/wc99ymFvW5NRRSjnVr4f/Xw02+n4Oi/v72bY5KkyZNVK9ePU2ZMiWvbdOmTZo5c6Y6dOigl19+OerzsrKyIkZcsrNNCUMAQCKUcyvXe50v6ZWiPLmLpPae45B7zcgMywBBr6NSt25d5/+rVpmFbfuY4/B90fTt29cJNOGbmf8CAIi/NlFCSq2ihJRKknKskPKme6mHkIJ0L/g2ePBgValSJe/WoIGZnQUAiKcH3JImYe+4C3Z+398TO0q6zWozGwl+yfuD4vHt0s/KlSud/9epUyfv4/Dxt98WvPPUrl27nBsAIP7MhfUdVts5RanBluWuWfb62C2DD6TiiMrChQu1YsUKnX766RHzTczqn+nTp/vVLQDIWO2ihJTqRQkpR0UJKUMJKUiBERWzPNlbF8VMoG3VqpXWrVunJUuWaNiwYfr3v/+tX375xQkuAwcOdGqqvPaa2c8bAJAswyX9n+f4FXfS7H7/1O3tzkkJmy1pYmL6iMyVsKVHnTp1CkUzZsyYvMcMGDAgtGLFCmdZ8vvvvx9q3rx5QpY3ceMc8D3A9wDfA/m/Bw6wlh2b25lF+V45JMqy4zp8j/E9prj//i7lfpCyzOUis/rHTKzdvHmz390BgJRxoqRPrLYqkvb7k/RySQd5js0Wyc/Ev39Ib0X9/c1ePwCQgUZLusxz/Lykf+7vSXXdfXq8xkpaFP/+AWEEFQDIIGY6yRar7TRJ0/b3RDNhpaXneJOkYZL+iH8fAS+CCgBkCBNIPrDaKrubDBaomqSbrLYJkubEv39AWhR8AwDE7r9WSBnlFnArNKR0jhJSTAl8QgqSiBEVAEhjZnLsxiiTaD8r7Elme+TbrbZ3Jc2If/+A/SGoAECa6uKWvvc6IEpRtwhtJZ0dpZ7+tvj3DygKggoApCFTc+1cz/Gjkm7c3zbJd1ptn0t6LzH9A4qKoAIAacSUvF8XZZBkVmFPOlxSd6vtEUnr498/IFYEFQBIE2bzwNejbDK4q7DlFGaYpaqn7UdJ4xPXRyBWBBUASANmrutZ1rSSOwp7QmNJl1ptIyWtSEz/gOIiqABACqslaY3V1kbSN4U9qafZJdZzbMLJU6m+oQrSFUEFAFLU+W7ttTBTJLaCpN0FPaG2pOustuckzU9cH4GSIqgAQAr6WNJJnuO7JfUv7AldJR3jOd4u6SFJexPXRyAeCCoAkELqSFpptbUsrFisqfh2S5S1y7MT0z8g3ggqAJAiLnZ3OQ7b7C5HLnBQ5BT35jVY0s7E9RGIN4IKAKSALyUd6znuK+m+gh5sJqr0sdqmSPo0cf0DEoWgAgABVl/SMquthaS5BT2hjVtQxcvMRdmSmP4BiUZQAYCA6iXpac/xKje4mNU9UX+am2GWMp42U472rcT3E0gkggoABNCPbmX7MDMfdmhBDz5U0kVW23BJvyeuf0CyEFQAIEAaSVpstTUrqNRJKbcuiqn6FjZP0guJ7SOQTAQVAAiIayU97jleJOmQggrGHiTpcqtttKSlie0jkGwEFQDwmRkYWehuv+MNLSMKW6fc3HNsaug/QQl8pCeCCgD4qKmkX602E1h+i/bgmpKut9pecC/3AGmKoAIAPrlZ0hDP8Ry3ymxUZ0tq6zne4xZSMf8H0hhBBQCSrLRbBt/sERh2maRnoj24sqRbrbZJkr5ObB+BoCCoAEASHSbpZ6utgaTl0R58gqQzrDYzirIjcf0DgoagAgBJYuqx3WvVY/NezclT3n2w14fuDcgwBBUASDBTLHajpEqetn9I+m+0Bx8t6TyrzUxk2ZTYPgJBRVABgAQyk2O/t9rquuXw86WZ29wNBcO+kfQ6bw8ym5nTBQBIgAFWSPnYrZmyKtoa5buskGLqohBSAEZUACDeyknaZbVdIOl/9gNNarlSUj1Pm6n89izvCRDGpR8AiKNjoqwcNsuQ19oPNOHkKqttTJSNfoAMx6UfAIiT+62QMtkdNMkXUrpZIWWDpLsJKUA0jKgAQAllSdpptXV167JFqC7pRqttvKQfeQuAghBUAKAETB2UmVZbDUnr7QeeKamj1XaPpN2cfqAwBBUAKKZHrT0CJ0YpgaKKkm632t6W9AWnHSgKggoAxOgASdusti7unJQI7d07vB6I8mQABSKoAEAMzPY7n1ptVe3CsWZ98p3Wg8yTpnCqgVix6gcAiugpK6T8113VExFSjowSUoYRUoDiYkQFAPbD7NGzxWo7XdJU+8++myVle9p+iFblDUAsCCoAUIhT7UAiqbKkrd6GgyVdYj3oSUkrObVASRFUAKAAz7m7HIc9Lely+0G9JDXyHC+VNJpTCsQLQQUALNn2vBNJJ9mTaOtIusZ60DhJCzidQDwRVADA4yxJ70YphbLd2/A3Sa08x+Y60MOS/uBUAvFGUAEA1ytWwbbhkm6w1yGbCbP2k77nFAKJQlABkPGqRSl5384uHnuapJOtB90raVfGnz4goQgqADLaX6NsHljBu8mgKUN7h/WA9yR9npz+AZmOgm8AMtbbVkh50C3glhdSjosSUsyDCClA0jCiAiDj1JS01mprI+kb70/GO93UEjYjyixbAAlHUAGQUc6XNMFqy5K0O3zQQlKPKNskr0tO/wBEIqgAyBgfSurkOR4o6T/hg1LuEp/qngf8LOml5PYRQCSCCoC0d6CkVVbbUe5WPI6Gki6zHjBK0rLk9A9AwQgqANLaRe4ux97abKYcyt5wwz8lNfU8YJW7T08ouf0EENBVP/3791coFIq4/fTTT353C0Aa+MIKKXe6Gwo6IaWWpBwrpJgHjyCkAEESiBGVH374QZ07d8473rNnj6/9AZDa6ke5anO4O+Ukr3jKsZ47TdG2+73DLACCIhBBxQSTVavsK8gAELtLJT3jOV4jqW54Gx6z22Bv6wmvSfqWMw0Ele+XfozmzZtr2bJlmj9/vp5//nk1bGhmtkWXlZWl7OzsiBsAyJ0c6w0pvd2JtE5IOTlKSBlMSAGCrpTfU8a6dOmiypUra+7cuapXr54zZ6VBgwZq2bKltmzZku/x5v6cHHNhOVKVKlW0efPmJPUaQJA0krTYamsmab75oLykvtadUyV9nLz+AcjPDDRs2rRpv7+/fQ8qtqpVq2rx4sW65ZZb9Mwz3r+N9o2olC9vfvLs+0LNaAxBBchM10h6wnNsAkuT8A+21pLOtZ7wsCT+pgFSJqgEYo6K18aNGzVv3jw1a2b+Hspv165dzg1AZivljpiYUBJ2XTi0lHH36DElZ8O+kvRG8vsJIA3mqHhVqlRJTZs21YoVK/zuCoCAOsSdd+INKQeHQ0pzSXdZIeUxQgqQqnwPKg8++KBOPvlkNW7cWB06dNDEiRO1d+9evfjii353DUAA3RSee+L6yR1dWWz+c62kiz13mgfmRNmBEEDK8P3Sz0EHHeSEkpo1a2rNmjX69NNP1b59e61dy08WAJF/VS2XVMfTdrmkp80HDSRdYZ0tc8cSziCQ6nwPKhdeeKHfXQAQcIdKmmu1HRQu6tbD3fE4zOxyPDxoywQApGxQAYDCmDmx91lzYo8zH9Rwdzv2eslbfhZAOiCoAAgks3BnvVtM1rt/4PPmgy6S2nvuMKMn95gy18nvJ4DEIqgACJwj3SqzXqYM/qpKkm6z7nhT0pdJ7ByAzFr1AwBeOVZI+cRd1bOqY5SQYjYSJKQAaY0RFQCB+WG0w73kE9ZN0gRTD6Wf9eCP3TL4ANIeQQWA70yl+2+sttqm/MlRkv5u3THUlLBOYucA+IqgAsBX97kre8Lek3RWaXenYzMnJWy2pIk+dBCArwgqAHxhrujstNrM/oGvm/r4/7LuGGEmqSSxcwACg6ACIOmOl/SF1VZd0oYr3CqzYb9Jyr+JOoAMwqofAEk1zAopr5tVPXWlDTlWSBlLSAHAiAqAJKkgabvVdrakd86X1NLTuMlNM2Z7ZAAZj0s/ABLOlED5zGrLriZtMVshe02QNIc3BMA+XPoBkFAjrZDyornU0zlKSDEl8AkpACyMqABIiIqStlptJ1eUPrndanxX0gzeBADREVQAxN0pkqZZbeXbSrvMpBSvByRt4w0AUDCCCoC4Gufuchw2opx07Z3Wgz53K7sBwH4QVADERba7YMfriMOln7pbjY9IWs9JB1A0BBUAJXampMme4z2lpawbpVBVT+OPksZzsgHEhlU/AEpkghVSbm4slfuPFVLM0h9CCoBiYEQFQLFUs67ghCRVvUTafLCncYWkp9w7AaAYCCoAYvYXSW94jufUllpeZz3oOUnzObkASoagAiAmb0r6s+e4bVdp1jGeBlMn/yFJezmxAEqOoAKgSGpKWus5XlJFanSL9aCJkmZzQgHED5NpAezXeVZIyTklSkgZTEgBEH+MqAAo1DS30qyxvoJUo4/1gCmSPuUkAkgMggqAqA6UtMpzPKqNdOU51oPMXJQtnEAAiUNQAZDPhZJecD/eUVaq0lfaXcbzgFmS3uLEAUg8ggqACDPNSh734zcOlc65yDpBwyX9zkkDkBwEFQCOepKWu+dibynpyOukubU8J2eeZ5gFAJKEoAJAl0ga456H6QdJHS+3TspoSUs5UQCSj6ACZLjvJB3lVrn/88XSO809d66R9AQl8AH4h6ACZKiGkn5zP55bU2pxvfWAF9zLPQDgIwq+ARnoKk9Iue5sK6TskTSIkAIgGBhRATJIKUm/SGpqNjauLNW/1XrAJElf+9Q5AIiCERUgQzSR9IcbUu47MUpIuY+QAiB4GFEBMsCNkoZJ2lReqtrXuvND9wYAAURQAdJ8yHSZpLqSnjta+pfZXdBriEkvPnUOAIqAoAKkqfamJoqkXWWkardJGyt47vxG0us+dg4AioigAqQhMyf2r5ImN5W6/NO609RFWe1TxwAgRgQVIM3+Qe82k2ZLSaX6W3culPSsTx0DgGJi1Q+QJjq5IeXNQ6Uydkgx9fEJKQBSECMqQBr4QNJppk5KTpQ7B5pdBn3oFADEAUEFSGFZknZK+qqeVMqUm/WaJukjnzoGAHFCUAFS1FmS3pV09DXS93WsOwe7CQYAUhxBBUhBMyUdWE0qdZN1x2xJE33qFAAkAEEFSCEHSNomqcf50sstrTuHStroU8cAIEEIKkCKOEfS2ApSqT7WHcslPeVTpwAgwQgqQAr4UdKLp0o1zBpkryclrfSpUwCQBAQVIMAqS1pbRqpwl3WH2Qb5bp86BQBJRFABAqqbpC6tpQrnWnc8L+lXnzoFAElGUAEC6DdJjXKk8fYdAySF/OkTAPiBoAIESFVJEw6RGv3LuuMNU9XNp04BgI8IKkBA9JT0v37SmabcrNcgSXt86hQA+CwQmxJee+21WrhwobZv364ZM2bo+OOP97tLQFJ9dqD0bI601RtSPpFk9u4hpADIYL4HlW7dumnIkCEaMGCA2rRpo9mzZ2vy5MmqXbu2310DEq6mpA6XSSdca91xv7vTIABkuFJ+T80zIyizZs3S9ddfn9uhUqW0ZMkSDR8+XPffb35aFy47O1ubNm1SlSpVtHnz5iT0GIiPf2RLz/eObCv7k7TnZc4wgPSXXcTf376OqJQrV07HHnuspkyZktcWCoWc4w4dOvjZNSChep2TP6ToEUIKAARqMm2tWrVUtmxZrVq1KqLdHLdo0SLqc7KyslS+fPmIRAakitpZ0pp+0jOetsprpS2P+dgpAAgw3+eoxKpv377OUFH4tmzZMr+7BBTJCSfmhhSvCqMIKQAQ2KCydu1a7dmzR3Xq1IloN8crV0bfwGTw4MHO9azwrUGDBknqLVCCf2U50medrfYcaQc5GwCCG1R2796tr776Sqeffnpem5lMa46nT58e9Tm7du1yJt14b0BQVW8p6T+Rbce95C47BgAEv+CbWZr87LPP6ssvv9QXX3yhm266SZUqVdKYMWP87hpQMjnSequpwgDpS0rgA0DqBJXx48c7NVPuvvtu1a1bV99++626dOmi1atX+901oHgamWU9kU03vyMNnSnt4JwCQGrVUSkp6qggSMr0lvZaC9FOu0eautuvHgFAMKVEHRUgbdTKvdTjDSnXz5SycggpAJDSl36AlPcPSc0im8Y/KHXb6leHACB9EFSA4qok6bbIpjN/lXY8L3XjrAJAXBBUgOL4k6R2kU0/PSYdtZbNjgEgnggqQCzKSbozsqnuZunJh6XDOZMAEHcEFaCo2rkjKR4fPyPd9pt0LmcRABKCoAIUZRF///zNf+Tk/gP6gzMIAAlDUAEKYzbx7hHZ9OL/pL0/sLYfAJKBoAIUJMp+PLvultr9IX3DWQOApCCoADazIfcVkU2Dp0h9Ps0dRUnpUs4AkGIIKoDX/7lVZj02DpZe3Jk7VQUAkFwEFcCoLunGyFPR62vp6UlSS0lzOEsA4AuCCtBd+YqgLH1YarCZURQA8BubEiJzHeBOmPWElA5LpFCONJ6QAgCBwIgKMtPpkk6KbPruCemo1dKhkn7xq18AgAgEFWTed/y/I5sq7ZK23Jv7MRNmASBYuPSDzNEmf0h5b1xuSLmHkAIAgcSICjK6BL65q4mkRX70CwCwXwQVpLdmkv4R2fTMa9Kl3+Z+zKUeAAg2ggrS112SykQ27Rgold8r3SnJnZYCAAgwggrST11JV0c23fWRdPe03I8PkrTMl44BAGJFUEF6uVJS/cimdfdJ1XfkfsylHgBILaz6QXqo4hZv84SU7j/kFm8zIeVmQgoApCRGVJD6/iapVWTTwmHSwRv2XQla5UvHAAAlRVBB6iovqW9kU8tV0vcj9h1zqQcAUhtBBanpZEmnRTZ9OVI6dkXux2Yu7UhfOgYAiCeCClJLGXfZscXMRQmrJen3pHYKAJAoBBWkjqMlnRfZ9MYL0l/m5X68UVI1XzoGAEgUggpSg2fEJGzvAKl0KPfjnpLGJb1TAIBEI6gg2A6WdElk0+NvSdfO2ndczR1NAQCkH4IKgut2SRUjm7YNkg7Yk/vxUkkNfekYACBZKPiG4KntXurxhJTen+dOmA2HlO6EFADICIyoIFgucS/3eKx5QKq1bd9xtqQtSe8YAMAPBBUEQ2VJt0Y2/Xme9OYL+45/knRE0jsGAPATQQX++7Ok4yOb5j0qNV+377irpElJ7xgAwG8EFfgnS1K/yKYqG6SNwyLbzFSV7UntGAAgKAgq8EcHSWdFNn32tNRxyb7jLyS1S3rHAABBQlBB8teZ/Sd/8x85kRsImgzzXjL7BQAIJIIKksfMhO0W2dR9vPTSj/k3Rd7F+wIAIKjAzxL4u++Wyv6x73iqpNN5SwAAHoyoILFM6djLIptqTpbWTo9sO0XSR7wXAAALQQWJc1P+7YzvuVfqZ13XKSfJLTgLAEAEggrir4akG6y2WVLorcimSW59FAAACkJQQXxdJOnQyKbmD0nztuRfnTyDcw8A2A+CCuKjorvbsdciacRY6eoo33R7Oe8AgCIgqKDkzpTU0Wp7XAqtiWwy2/ZczPkGAMSAoIKSfff822rbJrV+QPrGaj5W0tecawBAjAgqKJ7j3c0EvcZK4xZJ/4xSjDbEeQYAFANBBbExde77R2nOkTy12xyjJV3B+QUAlABBBUV3qLuqx+tVqe130kyr+ShJP3BuAQAlRFBBsUvga6D06l7pb1azd3NBAABKgqCCwtWXdKXVNlUq+7G022p+xC1GCwBAvBBUULBrJNWx2gZLJ+2UPraaD5M0j3MJAIgzggryqxZlaGS2pInSZLdsiheXegAAiWJWjvpm4cKFCoVCEbc77rjDzy7h/CghZYiUNTF3ibE3pNxLSAEApPuIyl133aVRo0blHW/evNnX/mSsCpL6WG3LJT0ldZb0vnXXISZoJrF7AIDM5HtQMcFk1apVfncjs50qqZPV9qSkldLn7gaCXlzqAQBkxKUfo0+fPlq7dq2+/vpr3XrrrSpTpkyhj8/KylJ2dnbEDcVUxl127A0ppmpbjlRhZe6lHm9IMdXyCSkAgIwZUXn00UedgLJu3Tp17NhRgwcPVr169dS7d+8Cn9O3b1/l5EQr6oGYtJZ0rtX2vKRfpb9IesO6q6GkpZxiAECSlYr3NiwmbJhRksK0aNFCc+fOzdd+6aWXauTIkapcubJ27dpV4IhK+fLl847NiMqyZctUpUoV5rcUVbScNyD3O+E7t6qsF6MoAIB4M7+/N23atN/f33EPKrVq1VLNmjULfcyCBQu0e7ddLkw64ogjNGfOHB122GGaN29eXL9QuDNg/2WdiUm52xpXkrTFusuMaw3hxAEAEqCov7/jfunHzDcxt+Jo3bq19u7dq9WrV8e7W+hnhqOs0zBI0h45K5InWHfVlcQUZwBAxs5Rad++vdq1a6dp06Y5SapDhw4aOnSonn/+eW3YsMGvbqWfOm6FWa9PJH2Q+6FZYnywdTeXegAAyvSgsnPnTvXo0cOZGGvmnJjibyaoDBnCxYa4ucydBet1v6TtUhVJG627rnFXJQMAoEwPKt98840zioIEMCu27YVTP0oan/vhPyQ9Z91dS9LvvBkAgIDxveAb4uwcSW2ibGu8PvdDE0ZqeO4yE2ipRAMACCqCSrrIcifMepk5zY/lflhd0jrr7ksljU1W/wAAKAaCSjo4Ubkb8niZ7ZOW5X54Re6WPRFMcGHKMgAg6Agqqb4Bwn8KL+hmyuaVs/YZbJCMvgEAEAcElVTVUrkFULxekvRz7oe1JdnVaHpIejlZ/QMAIA4IKmlWAt+4wZ0/65UdpfIsAABBR1BJJY3dGbBe70iaue/Q3g/B7KjUIhl9AwAgAQgqqaJ3lHXE90hyt0yqv2/ubB6zOfLryeofAAAJQFAJumqSbrLaZkh6d9/hHZLusx5SMbcALQAAKY2gEmQnSzrNantQ0taCL/V8Ken4ZPQNAIAkIKgEUXlJfa22XyU9HzldZZH1kD9FDrQAAJDyCCpB09qdXOL1sKTNkYt++lsPqWA2ekxG/wAASCKCSlCUcSebmFL4YV9JeiPyYfalng8lnZqM/gEA4AOCShA0l3Sx1faYu1ePq5mkX6yHnOoGFQAA0hVBxU+lJF0j6UBP23xJz+WfP3ur9dSsfSuTAQBIWwQVvzRwdwv0elrSksimHyUd7jl+U9Jfk9E/AAACgKDihx5Wudh1koZHTkAxOWap9bSOkqYnq48AAAQAQSWZargb8RSwkWDY5ZJGeY5XSDpI0h/J6CMAAAFCUEmWLpLae45Dbgn8PZEPM5nlMM/xTVE2GAQAIFMQVBKtkqTbrLY33RKyKryAW1NJCxLdPwAAAoygkkhmUsmZVtv9+Tfhuc5djexd+NM8Ss0UAAAyDUElEcza4X5W28eSpuZfnbxYUkNPm1mt/GRCOgUAQOohqMTbUZL+brUNlbQx/2Uds32PV6P8q5MBAMhoBJV4KS2ptzsnJWy2pIn5H2oe9pDn+HtJR8etIwAApA+CSjwcIulfVtsISavyZ5nVkmp62npJGhOXTgAAkH4IKiV1hVudLew3Sc/kf5ip7/aT1VbfrZECAACiI6gUV11JV1ttY6OsMZZ0p6RBnuOZVkkVAAAQHUGlOM6X1NJzvEnSsPylY8u6dx3gaTObJL9QrE8KAEDmIajEoppbKtZrgqQ50Rf/fGe11XHnqAAAgKIx8ztRFJ2jhJR7ooeUu62QMs2tmUJIAQAgNoyo7E9FSbdbbe9KmpH/oeUk7YpyleiVGN8UAACQi6BSmLaSzrbaHpC0Lf9D20j6ymqrJen3Qj8BAAAoDJd+CquN4g0pn0vKiR5SHrRCytvupR5CCgAAJcOISkHMdZywRyStz/+Q8pJ2WG3nSHqjhG8KAADIRVApyFxJAyXtjX53uyjTVKpL2lDgCwIAgFhx6acwBYSU4VZIecW91ENIAQAgvhhRicEBUaaonCXpvfi+JwAAwEVQKaITJX1itVWRtLmoLwAAAGLGpZ8iGG2FlOfcSz2EFAAAEosRlUJUkrTFajvNrTQLAAASj6BSALMvz0qrrbKkrYl/TwAAgItLPwW4wPPxU+6lHkIKAADJxYhKAZ53/z9T0qzkvR8AAMCDoFIAUxPlsYLuBAAAScGlHwAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAEFgEFQAAkHlBpV+/fvrss8+0detWrV+/PupjGjZsqDfffNN5zKpVq/TAAw+oTJkyieoSAABIMQkroZ+VlaUJEyZo+vTpuuyyy/LdX7p0ab311ltauXKlOnbsqHr16mncuHHavXu37rzzzkR1CwAApJhQIm89e/YMrV+/Pl97ly5dQnv27AkdeOCBeW1XXXVVaMOGDaFy5coV+fWzs7NDhvl/or8WbpwDvgf4HuB7gO8BvgcUl3NQ1N/fvs1R6dChg77//nutXr06r23y5MmqWrWqjjzyyEJHarKzsyNuAAAgPfm2e3LdunWdeSle4WNzX0H69u2rnJycfO0EFgAAUkdRf2/HFFQGDx6sPn36FPqYFi1aaO7cuUoU04chQ4bkHZu5LebzLVu2LGGfEwAAJC6wbN68OT5B5eGHH9bYsWMLfcyCBQuK9FpmEm3btm0j2urUqZN3X0F27drl3MLMF9egQYNCv8hYTpYJPPF6vXTAOeGc8H3Cvx1+nvDzNZG/Y5YvX17oY2IKKmvXrnVu8WBWA5nVPbVr19aaNWuctjPOOEMbN27Ujz/+GNNr7e+LjJUJKQQVzgnfJ/zb4edJYvAzlvMRVpTftQmbo2JqpNSoUUONGjVyaqO0atXKaf/111+duinvvfeeE0iee+453X777c68lEGDBunxxx+PGDEBAACZLSFLrcaMGROKplOnTnmPadSoUeitt94Kbd26NbR69erQgw8+GCpTpoxvS79Y6sw54fuEfzv8POFnLL9zFLQl2L53IDC3rKysUP/+/Z3/+92XoNw4J5wTvk/4t8PPE36+ysffQ6XcDwAAAAKHTQkBAEBgEVQAAEBgEVQAAEBgEVQAAEBgEVQK8Prrr2vx4sXavn27U1Bu3LhxTrn+TNW4cWONHj3aqTy8bds2px6O2XOpXLlyymT9+vXTZ5995tQGWr9+vTLRtddeq4ULFzr/VmbMmKHjjz9emeykk07SpEmTnCrXoVBIXbt2VSYz26588cUX2rRpk7Of28SJE3XooYcqk1199dWaPXu2U+DU3D7//HN16dLF724FFkGlANOmTVO3bt102GGH6e9//7uaNm2q//3vf8pUZg+n0qVL66qrrnJ2t7755pudf2z33nuvMpnZzXvChAkaMWKEMpH5N2L23howYIDatGnj/PA1u6CbitOZqlKlSs55uO666/zuSiB06tTJKeTZvn17p/q4+ePGFPysWLGiMtXSpUudAHfsscfquOOO09SpU50/jo844gi/uxZYvtfqSIXbX//619DevXtDZcuW9b0vQbndeuutofnz5/vejyDcevbsGVq/fr3v/Uj2bcaMGaHhw4fnHZcqVSq0dOnS0B133OF734JwM7p27ep7P4J0q1WrlnNeTjrpJN/7EqTb77//HurVq5fv/VAAb4yoFEH16tV18cUXO8Nze/bsSXx0TBFVq1bVunXr/O4GfGL+MjZ/EU6ZMiWvzVzqMMcdOnTgfUGBPzcMfnbkMiPV3bt3d0bizB54yI+gUoj77rtPW7Zscf5BmT2LMv1as5e5FHb99ddr5MiRfncFPqlVq5bKli3rzDvwMsdm7y7AVqpUKQ0bNkyffvqp5syZk9EnqGXLls6GfDt37tSTTz6pv/3tb/rpp5/87lYgZVRQGTx4sPMXX2E3Mycl7MEHH9QxxxzjXFfdu3evM6E208+JUb9+fb377rvO3AwzwTbdFOecANg/M1fF/ILu0aNHxp+uuXPnqnXr1mrXrp0zx+3ZZ5/V4YcfnvHnJZqMKqFv/gKsWbNmoY8xq1p2796dr71BgwbOBCgzpG1WNmTqOTErnz788EPnHFxyySXOL+10U5zvk549ezp/KZrLhJl06cesADv//POdiYBhY8eOVbVq1XTuuecq05l/H+Y8eM9Ppho+fLgzKn3yySdr0aJFfncncN5//33Nnz/fWaSASGWVQdauXevcinsd0Shfvrwy9ZyYkRSzGuqrr77SpZdempYhpaTfJ5nEBDXzvXD66afn/SI2Q/vm+LHHHvO7ewhYSDGXNk455RRCSiG/Y9Lt90u8ZFRQKaq2bds6tSDMdVRTG8PMxxg4cKBTOyRTJzuZkGJGUkxtmVtvvTVi+ak9RyGTNGzYUDVq1HDmMJUpU0atWrVy2s33iqmtku7M0mQzZP3ll186tTJuuukmZ1LgmDFjlKnM19+sWbO84yZNmjjfF2au25IlS5SJl3suuugiZzTFzMmoU6eO027qh+zYsUOZyJR1eOedd/Tbb78pOzvbOT8mxJ111ll+dy2wfF96FLRby5YtQx988EFo7dq1oe3bt4cWLFgQeuKJJ0L169f3vW9+Lr8tiN998/M2ZsyYqOekU6dOvvctWbfrrrsutGjRotCOHTuc5cpt27b1vU9+3sx7H435XvG7b37cCmJ+pvjdN79uo0ePDi1cuND5N7Nq1arQ+++/H+rcubPv/VJAbxk1RwUAAKSWjFr1AwAAUgtBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAABBZBBQAAKKj+H1kYL0Xn2KgMAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# actual vs predicted\n", "\n", "plt.plot(xs, ys, c=\"r\", label=\"actual\")\n", "plt.plot(xs, lin_reg_model(trained_params, xs), c=\"g\", label=\"predicted\")\n", "plt.legend()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So that's it. You now have an elementary understanding of a training loop using jax, (and or, flax) and optax. Let's recap a bit. You initialise your params, create the optimiser, initialise the optimiser states and then during training, update the optimiser state and params using optax. \n", "\n", "Also: make sure to check what you can `jit` and what you can't. " ] } ], "metadata": { "kernelspec": { "display_name": ".venv", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.13" } }, "nbformat": 4, "nbformat_minor": 4 }