{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# その他の回帰手法\n", "\n", "
\n", " このページのオリジナルのipynbファイル\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ニューラルネット回帰\n", "\n", "一つの説明変数、一つの目的変数の非線形フィッティングにニューラルネット回帰を使うことはあまりないだろうが、機械学習の手法を回帰に使う事例として、これまで取り上げてきたデータ例をもとに試してみよう。\n", "\n", "(カーネル法をいったん忘れて)基底関数の線形結合で目的変数を表す:\n", "$$\n", " y = \\sum_{j=1}^D w_j \\phi_j (\\boldsymbol{x}) \\tag{1}\n", "$$\n", "\n", "これまで、訓練(トレーニング)データから機械学習の幾つかの方法を使って、係数$\\{w_j\\}$を決定することを行ってきた。\n", "\n", "ニューラルネットの回帰手法は、これまでのフィッティングのイメージを引きつで説明すると以下のようになる。\n", "\n", "- (1)式の出力を、複数個考える:\n", " $$\n", " a_i = \\sum_{j=1}^D w_j^{(1)} \\phi_j (\\boldsymbol{x}) \\tag{1}\n", " $$\n", " これが隠れユニット(神経細胞に相当)$i$への入力となる。\n", "\n", "- 隠れユニットは、その入力値がある一定値を超えると発火すると考える。(これがニューロンのイメージを引き継いだ考えである。ただし、ここで階段関数を用いるのではなく、もっと平坦な関数を用いることもある。)その関数を$h(a_i)$とする。\n", "\n", "- 隠れユニットの$i$番目のノードの「発火」は重み$w_{ji}$をかけて次の層の$j$番目のノードに伝えられる。\n", "\n", "単純のために隠れユニット層を1層とすると、\n", "\n", "$$\n", " y = \\sigma \\left(\n", " \\sum_{j=1}^M w_{j}^{(2)} h\\left( \\sum_{j=1}^D w_j^{(1)} \\phi_j (\\boldsymbol{x}) \\right)\n", " \\right)\n", "$$\n", " \n", "関数$\\sigma$, $h$は、外から与えるものであり、階段関数、線形関数、ロジスティック関数などが用いられる。\n", "\n", "PRMLからの図を引用する(図5.1) この図では出力は$M$個書かれているが、上の式では、出力(目的変数)一つだけにしたことに注意。\n", "\n", "\n", "\n", "トレーニングデータを用いて、ノード間の結合係数$w_{ij}^{k}$をどのように決めるかがニューラルネットの本題であるが、ここでは省略する。\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## scikit-learnのニューラルネット回帰\n", "\n", "scikit-learnにはニューラルネット回帰のモジュールMLPRegressorが提供されている。(MLPはMulti-layer Perceptronの略)\n", "\n", "今まで使ってきた$\\sin$関数に乱数を加えたデータと多項式(べき)基底関数を使って、MLPregressorを試してみよう。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# データ作成用関数\n", "import random\n", "def make_data_by_sin_gaussian(x):\n", " # y=sin (x)を計算し、ガウス分布に従うノイズを加える\n", " y = np.sin(x)\n", " e = [random.gauss(0, 0.2) for i in range(len(y))]\n", " # e = np.random.randn(len(x))*0.2\n", " y += e\n", " return y\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhW5Z3/8fc3e4CQkJCNkAQQCItYloAiKoggVmml0FKrrY5L7VSnznQqrbZTf7bOVGaYbk47rXS6aGtrVRBwKyrIoigaFgn7ImsSQgKEsCRku39/JNiA2ch2nufk87ourovnnOfJ+brkc87zPfd9H3POISIi/hXidQEiItKxFPQiIj6noBcR8TkFvYiIzynoRUR8LszrAi7Uu3dv169fP6/LEBEJKuvWrSt2ziU2tC/ggr5fv37k5OR4XYaISFAxs/2N7WtT68bMPm9mz5nZgUb2P2BmH5jZRjN7sC3HEhGR1mlrj74IuA+IuHCHmU0AvgRMAMYBM8wsu43HExGRi9SmoHfOrXTOFTeyezrwe+dchXOuAvgdcHNbjiciIhevI0fdJACH670uAJIaeqOZ3WtmOWaWU1RU1IEliYh0PR0Z9IWcH+wpdds+wTk33zmX7ZzLTkxs8KaxiIi0UkeOulkM/MTMngJqgDsA39yQXbQhj3lLd5BfUkafuGjmTMtixqg0r8sSEfmEdg96M1sB3OKcyzGzJcBaoBp41jnni3GTizbk8fDCXMoqqwHIKynj4YW5AAp7EQk47dK6cc6l1Pv7JOfc4bq//7dzbrRzbqxz7sftcaxAMG/pjo9D/pyyymrmLd3hUUUiIo3TEgitkF9SdlHbRUS8pKBvhT5x0Re1XUTESwr6VpgzLYvo8NDztkWHhzJnWpZHFYmINC7g1roJBuduuGrUjYgEAwV9K80YlaZgF5GgoNaNiIjPKehFRHxOQS8i4nPq0UuTtNSDSPBT0EujtNSDiD+odSON0lIPIv6goJdGaakHEX9Q0EujtNSDiD8o6KVRWupBxB90M1YapaUeRPxBQS9N0lIPIsFPrRsREZ/TFb006aOiU6zZc5SSMxWUlldRUVXD0NQYRmX0YmBiD0JCzOsSRaQZCnr5hOOnK/hrzkFe+jCfLfmlH2+PCg8h1IzTFbVj62Ojw7llXDp3X9WfpJio836GZtSKBA4FvXysoqqGp9/dxxPLdlFaXsXI9Di+P30Y1w9LJqlnJJFhoTjn+Kj4NOv3H2fFjiJ+s+oj/vDOPr44Np0HrhtE7x6RmlErEmDMOed1DefJzs52OTk5XpfR5azeVcQji7ewt/g01wxO5Hs3DiUrJabZz+0tPs2TK/ewYP0hekaF8x+fG8FjL28lr4FJVWlx0bzz0OSOKF+kyzOzdc657Ib26Yq+iztTUcXc17bz9Lv7GZDYnd/fOZZrs5Ja/Pn+vbszd9Zl3DmhP//63Eb+8U/rGn2vZtSKeENB34VtPFjCN/+6kb3Fp7n7qv7MmZZF1AUTpFoqKyWGF++bwC/e2s0Ty3Y1+B7NqBXxhoI+iLTXDc6q6hp++dYenli+i+SYSP58z+VcObB3m+uLCAvhX6cOprq6hl+u2HPePs2oFfGOgj5ItNcNzgNHz/DN5zaybv9xbh7Zhx/efCmx0eHtWuucG4YQ2y2c//rbDqpqHL26hfP/PjNcN2JFPKIJU0GirUsGV1XXMH/VHq7/2Up2Fp7k57eM5Oe3jGr3kD/n3msuYd2/TWVc/3hOlFVSVRNYN/1FuhJd0QeJtiwZvOlQCd99MZfNeaVMGZrMYzOGkxrb8f3y2G7hPHXnOO79Yw4PPv8hZ6uque3yzCY/o/H3Iu1PQR8k+sRFNzhksakbnB8VneLHb+zklU0FJMZE8r+3jebTl6Zg1nmzWaMjQvnN7dnc98x6vvfiZqprHLeP79fgezX+XqRjqHUTJC5myeDth0v59gsfMvWnq3hr+xEemDyQZd+ayI0jUjs15M+JCg/l118ew5ShyTyyeAuLNuQ1+D490UqkY+iKPkg0t2RwZXUNy7cf4ak1+1iz5yhR4SF85YpM7r92IIkxkV6WDtSOyPnFraP4h9+/z7ee/5Ce0WFMHpJ83nv0RCuRjtHlgz6YesIXLhnsnGNL/gkWrMtj8cY8jp6uIDU2iu/cMIQvjUsnrluEh9V+UlR4bRvn1t+s5et/Ws8f776ccf3jP97fmvaUiDSvSwd9sPaEDxw9w5IP81i0MZ/dR04RERrClGFJzBrdl4mDEwkLDdyOXExUOH+4cyxfePJd7v1jDovum0C/3t2B2vZU/f8eoPH3Iu2hS691M2Hu8qBZk+X46Qpezi1g0YY81u0/DsC4fvF8dmQfbhqRSq/ugXX13pz9R08z45fv0Kt7BC9+fQKx3WqHeQbTNyyRQKK1bhoR6D3hquoaVu8q5vl1B3ljayGV1Y7ByT349g1Z3DwyjbQgbmlkJnTnya9kc9v/vcf9f17P7+8cS3hoiJ5oJdIBunTQB2pPOK+kjL++f4Dncg5xuLSc+O4RfOWKfswak8aw1J6ejJzpCOP6x/Ojz41gzgubeOzlrfzw5ku9LknEl7p00AdST7imxrFyZxF/fG8/K3YcwQGTBify6GeHM3lIEhFhgdt3b4svZKez68gp5q/6iDGZvbh5pK7mRdpblw765oYsdobjpyt4Lucgf1q7n4PHykiMieT+awfyxbHp9O3VrdPq8NKcaVlsOHCchxfmMrxPLAOTenhdkoivdOmbsV7anHeCp9/dx+KN+ZytqmFc/3huH5/JtOEphAfwqJmOcvhEOTc9sZqEHhEsvv8qoiNat1yySFelm7EBoryymtc2F/D0u/vZcKCE6PBQZo3py+3jMxmS0rPDjhsMI1lSYqP4+S2j+Mrv1vLI4s3M+8KnvC5JxDcU9J1gb/Fpnn3/AM+vO8Sx0xUM6N2dR6YPY9aYvh22euQ5wTRX4KpBvfmnawfyP8t3M2VYMtOGp3hdkogvtCnozWw28CAQCqxwzn3rgv0rLvjIt51z77flmMGivLKapVsO89cPDrJmz1FCQ4ypQ5P5yvhMrrwkodNGzjS1fkygBT3AA9cNYvn2I3x3YS5jMnvRu4f3yzeIBLtWB72ZZQKPAeOAUuBZM5vlnFtQ722RzrnxbawxaDjn2HiwhAXrD7FkYz6l5VWkxUXz4PWDmZ2dTlLPqE6vKdDnClwoPDSEn8weyWf+522+92Iuv/7yGN8MJxXxSluu6G8AFjjnTgCY2ZPAncCCutdhQKyZPQekACuBR51z1Rf+IDO7F7gXICMjow0leWNP0SmWbMxn8cY89h09Q2RYCJ++NIUvZKczfkACISHeBVWgzhVoSlZKDN+6fjCPv7adFzfkMXN0X69LEglqbQn6BOBwvdcFQFK91z2oDfeHqb3inw/cAzx54Q9yzs2v2092dnZgDQNqgHOOHYUn+dvmw7yWe5gdhScxg/EDErhv0kBuGJFCz6iO7b23VCDNFbgY91w9gDe2FvLoki1cMzhRLRyRNmhL0BcC/eu9TqnbBoBzrgT4+rnXZrYQmEUDQR8MyiqqeX/fMZZvK2TZ9iMcOl6GGYzNjOfRzwzjhktTSYnt/NZMcwJhrkBrhIYYc2eN4NM/X82PXt3GT2aP9LokkaDVlqB/FXjTzP7TOXcSuAtYdG6nmaXUbXvc1Q7WvwFY35ZiO1NFVQ25eSdYu/co7+wu5oN9x6moqiEqPISrBvbm/msHct2QJE/67hcrWNePGZgUw1evHsD/rtjD7Ox0rhiQ4HVJIkGp1UHvnCswsx8Bq8ysAljtnFtQN9LmFmqv7nsA683sFLCRuvZMICo6eZYPD5aw4eBxNhwoYcOBko/bHUNSYrj9ikyuHpzI5f3jiQrXZJ7O8o3Jg1i8MZ/vL9rMKw9c7dulIEQ6UpebGVtRVcNHxafYWXiKXYUn2ZJfypb8ExSWngUgLMQYkhpDdmY8VwyIZ2y/eBLUH/bUm1sLuefpHL5zwxC+PukSr8sRCUhdbmbsyfJKCk6Uc/DYGQ4eO8OBY2XsLT7FvqNnOHDsDNU1tSe30BDjksTuXHlJb4b36cmn0uO4tE+spt8HmCnDkpk6LJknlu1i1ui0oGiXiQQS3wT9C+sOMX/VHgpKyjl5tuq8fdHhofTv3Z1hfXpy04hUBiX3YFBSDAMSu6sNEyS+d+NQpv50JT99cxePzxzhdTkiQcU3Qd8jMozMhO6MH5BAalw0qbFRZMR3Iz2+GwndIzTpJsj1692d2y7P5Ol393H3Vf0YmBTjdUkiQaPL9egleB07XcHE/3qLywfE8393jPW6HJGA0lSPXkMYJGjEd4/g69dewpvbjvDeR0e9LkckaCjoJajcNaE/qbFRPP7qNgLt26hIoFLQd7JFG/KYMHc5/R96hQlzl7NoQ57XJQWVqPBQvjl1MB8eOsGybUe8LkckKCjoO9G5teHzSspw/H1teIX9xZk5Ko3MhG78bNlOXdWLtICCvhM1tTa8tFxYaAj/dO1ANueV6qpepAUU9J0o2NaGD2Sf01W9SIsp6DtRY2vAB/La8IFKV/UiLaeg70RzpmURfcFM3GBYGz5Q6apepGUU9J1oxqg0Hp85grS4aAxIi4vm8ZkjgnIJ4UAQFhrC/XVX9St3FnldjkjA0sxYCWoVVTVMnPcWGfHd+OvXuszjiUU+QTNjxbciwkK4+6r+rN17jPUHjntdjkhAUtBL0PvSuAxio8P59Yo9XpciEpAU9BL0ukeGccf4TN7YVsjuI6e8Lkck4CjoxRfuuLIfkWEhzF+lq3qRCynoxRcSekQyOzudFzfkcfhEudfliAQUBb34xlevHkB1jeOpd/d5XYpIQFHQi2+kx3fjhktTeOa9/Zy+4HGSIl2Zgl585Z6rB1BaXsXzOQe9LkUkYCjoxVdGZ/RidEYcv3tnH9U1gTUZUMQrCnrxna9ePYADx87wxtbDXpciEhAU9OI71w9PIT0+mt+s3ut1KSIBQUEvvhMaYtw1oT/r9h/XsggiKOjFp2ZnpxMTFcbv39nndSkinlPQiy91jwzjlrHpvJpbQMEJPcFLujYFvfjW7eP74Zzj6Xf3e12KiKcU9OJb6fHduH5YCn95/wBlFdXNf0DEpxT04mt3XdWfkjOVvLghz+tSRDyjoBdfG9uvF5em9eR37+zVc2Wly1LQi6+ZGXde2Z/dR06xelex1+WIeEJBL76xaEMeE+Yup/9DrzBh7nIW1bVrpn8qld49Inj63X2e1ifiFQW9+MKiDXk8vDCXvJIyHJBXUsbDC3NZtCGPyLBQvjQug2Xbj3Dg6BmvSxXpdAp68YV5S3dQVnn+yJqyymrmLd0BwK2XZxBixp/WaqildD0KevGF/JKGJ0Wd254aG80Nw1P46wcHNdRSPqGxtp9fKOjFF/rERTe7/fbxmZwoq2TxRn/9EkvbNNX28wtfBL3fz8bSvDnTsogODz1vW3R4KHOmZX38elz/eIakxPDUu/s11FI+1lzbzw/aFPRmNtvM3jezdWb24wb2P2BmH5jZRjN7sC3HakxXOBtL82aMSuPxmSNIi4vGgLS4aB6fOYIZo9I+fo+ZcceV/dhWUMoH+7SqpdRqru3nB2Gt/aCZZQKPAeOAUuBZM5vlnFtQt38C8CVgQt1HlpvZCudcThtrPk9TZ+P6v+TifzNGpTX733zGyDR+9Oo2nlm7n3H94zupMglkfeKiyWsg1BtrBwajtlzR3wAscM6dcLXfg58EZtTbPx34vXOuwjlXAfwOuLkNx2tQVzgbS/uJjghl1ui+vJZ7mGOnK7wuRwJAS9p+wa4tQZ8A1H9WWwGQdBH7P2Zm95pZjpnlFBUVXVQRLbkJJ1LfrZdnUFFdwwvrah8grns8XVtL2n7BrtWtG6AQ6F/vdUrdtvr7k5rY/zHn3HxgPkB2dvZF3SWbMy2Lhxfmnte+8dvZWNrX4OQYsjN78Zf3D5LYI5Lvvrj54/9/zt3jAXz1iy5Na0nbL5i15Yr+VeBzZhZT9/ouYHG9/YuB280s3MxCgTuAJW04XoO6wtlY2t9tV2Swt/g0//HKNt+PuBBp9RW9c67AzH4ErDKzCmC1c26Bma0AbnHO5ZjZEmAtUA082943Ys/x+9lY2t+nL03lBy9tpbiRPr3u8YiftKV1g3PuGeCZC7ZNqvf3/wb+uy3HEOkIUeG1N2V/+/beBvfrHo/4iS8mTIm0xq2XZwAQFmLnbdc9HvEbBb10WZck9mBsv17Ed4+gT2yU7vGIb7WpdSMS7GaN7stDC3N58b4rGZXRy+tyRDqEruilS7vxslSiwkNYsP6Q16WIdBgFvXRpPaPCuWF4Cks25lNeqeWLxZ8U9NLlfX5MOqXlVby5rcH5fCJBT0EvXd74SxJIjY3ihXVq34g/KeilywsNMWaOTmPVziKOlJZ7XY5Iu1PQi1A7+qbGwYta0Ex8SEEvAgxI7MGn0uN4aVO+16WItDsFvUid6SNS2ZxXyr7i016XItKuFPQidW68LBWAV3ILPK5EpH0p6EXqpMVFMzojjpc3KeiDlR4i0zAFvUg9N13Wh20FpewpOuV1KXKRFm3I4+GFueSVlOH4+0NkFPYKepHz3DgiBYBXdVUfdOYt3aGHyDRCi5qJ1JMaG83Yfr14JbeAb1w3yOty5CI09rCYC7dXVdewt/g0eSVlFJ08S9Gps5wqr6KsspqyimoqqmuornFU1ThqahzOQY1z1H/GqbuoB5623JevyGBSVoOP1m4TBb3IBW4akcqjL21l95GTDEyKaf4DEhD6xEWT10DY94mLZmt+KQvXH2L9geNsLSilvLLmvPeEhxpR4aFEhYcSGRZCWIgREmKEmhFihtU9ssDs788uOP8pBu3j9NmOWW9JQS9ygU+PSOUHL2/l5U0F/MsUBX2wmDMti4cX5p7XvgkPNUJDjBufWE1EWAgj+8Zx67hMRvTtSUZ8NxJ7RNE7JoJuEf6OQn//04m0QnLPKMZk9OL1LYX8y5TBXpcjLXTuYTHzlu4gr6SMyLAQzlbVEBUewiPThzFzdBpx3SI8rtIbCnqRBlw3NJn//Nt28kvK9PzYIDJjVBo9o8N48PlNlFVU85+zRjA7O/28lktXpFE3Ig2YMrT2htjy7Uc8rkQuxiubCrjrDzmk9IzipW9cxRfHZnT5kAcFvUiDBib1ICO+G8u0Rn3Q2Fl4kjkvfMjojDgW3nclA5N6eF1SwFDQizTAzLhuaBLv7DnKmYoqr8uRZpSWV/KPf1xHt4gwfvXlMUSFh3pdUkBR0Is0YsrQZCqqanh7V7HXpUgTnHM8+NyH7D92hl/eOorknlFelxRwFPQijRjXP56YqDCWbVOfPpC9tKmA17cW8vCnh3D5gASvywlICnqRRoSHhjBxcCLLth+hpqaDpkJKmzjn+NWKPQxM6sFdE/p7XU7AUtCLNGHK0GSKT51lU94Jr0uRBqzeVcy2glLuvWYAISEaXdMYBb1IEyZlJRIaYhp9E6CeXLWH5J6R3Dyyj9elBDQFvUgT4rpFMCajF2/tUJ8+0OQeOsE7u49y91X9iQzTKJumKOhFmjExK5HNeaUUnTzrdSlSz69X7SEmMowvjcvwupSAp6AXacbEwYkArNpZ5HElcs7+o6d5LbeA267IJCYq3OtyAp6CXqQZw1J70rtHJCsV9AFj8cZ8HPAPV/bzupSgoKAXaUZIiDFxcCKrdxVRrWGWAeHNbYWMSo8jJVaTo1pCQS/SAhOzEjl+ppJNh0q8LqXLKywtZ9OhE1w3NNnrUoKGgl6kBa4e2Bsz1L4JAOdmKk9R0LeYgl6kBXp1j+BTfeMU9AFg2bZC0uOjGZys1SlbSkEv0kKTshLZeLCE46crvC6lyyqrqObt3cVcNyRZ68xfBAW9SAtNHJyIc7B6t1az9Mrbu4s5W1XD1GFq21wMBb1IC13WN45e3cJZoVmynnlzayExkWGM7RfvdSlBpdXPjLXa700/AiYDkcA859wzF7wnDDgMbK63+XrnnL77StAJDTGuGZzIyh21wyxDtYhWp6qpcSzbfoSJWYlEhOka9WK05d/WrcAg4ArgGuB7ZpZ6wXvSgdedc5Pq/VHIS9CaOiyZo6crWH/guNeldDkfHiqh+NRZjbZphbYE/XRgvqtVCrwA3HjBe/oBSWb2mpmtNrNb2nA8Ec9NHJxIeKjxxlatZtnZVu8qxqz2prhcnGaD3swmm9mKC/8ACdS2Zc4pAJIu+PgZYAW1J4XpwBwzG9bAMe41sxwzyykq0vA1CVwxUeGMv6Q3b2wtxDnNku1MH+w7RlZyDHHdIrwuJeg0G/TOueUXtF4mOecmAYWcH+wpddvqf3atc+7fnXPVzrkTwDJgTAPHmO+cy3bOZScm6mwtgW3qsGT2Fp9mT9Epr0vpMqprHBsOlJDdr5fXpQSltrRuFgN3A5hZN2Am8Fr9N5jZhHPtGjOLBCYBG9pwTBHPTa3rEb+u9k2n2X64lFNnqzTappXaEvQLgHwzywFWAnOdcwVmNtLMnq17zzZgppl9QG0LZ75zbnPDP04kOKTERnFZ31j16TtRzr7am9/ZCvpWafXwSlfboPxWA9s3ArfU/f0YMLvV1YkEqKlDk/nJmzs5crKcpBitoNjRPth3jNTYKNLior0uJShpMKpIK0wdnoxzf19gSzqOc46cfcd1Nd8GCnqRVshKjiE9PprXtxxu/s3SJnklZRwuLSc7UzdiW0tBL9IKZsaNI1JZtauYwyfKvS7H19btP9efV9C3loJepJVuHZdBjXM8+8EBr0vxtQ/2HaNHZBhDUnp6XUrQUtCLtFJmQncmDk7kL+8foLK6xutyfCtn33FGZcRpbaE2UNCLtMGXL8+ksPQsb2qoZYc4UVbJjsKTGj/fRgp6kTa4dkgSaXHR/PG9/V6X4kvrDxzHOXQjto0U9CJtEBpi3Hp5Bmv2HGX3ES2J0N7W7TtOaIgxMiPO61KCmoJepI2+ODad8FDjmbW6qm9vH+w7xqV9etItotVzOwUFvUirLdqQx4S5yxn7728SFhLCn9ceIK+kzOuyfKOiqoaNB0s0UaodKOhFWmHRhjweXphLXkkZDiirrOZsVQ1ffSqnRcsXnztJ9H/oFSbMXc6iDXkdX3SQ2Zx/grNVNYzV+Pk2U9CLtMK8pTsoq6z+xPatBaUs+TC/yc9eeJLIKynj4YW5CvsL5Ow7BsCYTF3Rt5WCXqQV8pto0Ty6ZAtHT51tdH9DJ4myymrmLd3RbvU1JNi+RXyw7zj9ErqRGBPpdSlBT0Ev0gp9GllFMSkmklNnq3hkyZZGWziNnSSaOnm0VbB9i3hx/SGWbStk39EzQXFSCnQKepFWmDMti+jw0PO2RYeH8t0bh/IvUwbzyqYCvrNgE1UNzJht7CTR2Pb24NW3iNZYtCGPhxbmUlN3ngz0k1IwUNCLtMKMUWk8PnMEaXHRGJAWF83jM0cwY1Qa9026hH++bhDP5Rzi68+sp/yCgG3sJDFnWlaH1evFt4jWmrd0B2erzj9BBupJKVhocKpIK80YlcaMUWmf2G5mfHPqYOK7R/DoS1v48v+tZe6sEQxMivn4c1AbaPklZfSJi2bOtKwGf1Z76RMX3eDQz478FtFawXRSChYKepEOcseV/YjvHsHDC3O5/qer+OLYDL45ZRBJPaMaPUl0lDnTsnh4Ye557ZuO/hbRWsF0UgoWat2IdKDPfKoPK+dM4vbx/Xhh3UGu/q+3uP/P63lzayEVVZ234mVTraZA87VrBnxiW6CelIKFtWRyR2fKzs52OTk5Xpch0u72Hz3Nb9/ey8ubCjh2uoK4buFcPSiRqwf15ppBiaTE6tmzAK/lFvD1Z9aT2COS4lNnO6W15Qdmts45l93QPrVuRDpJZkJ3fnjzpXx/+jBW7yri5U0FrN5VzEt1E6wGJHbnigEJXDEggfEDErrs+PE1e44SFR7COw9NJiJMTYf2oKAX6WThoSFMHpLM5CHJOOfYfvgkq3cV8d5Hx1iyMZ8/r619YtWgpB6MvySBCQN7M/6SBHpGhXtcecc7W1XNS5vyuW5oskK+HSnoRTxkZgxN7cnQ1J7ce80lVFXXsCW/lHc/OsqaPUd5PucQT7+7v3ap3vQ4Jg5OZPKQJIb36YmZ/5649PqWQkrOVPLF7HSvS/EV9ehFAlhFVQ3rDxzn7V3FrNpVxKZDJ4DaGbjXDU3i+mEpjL8kgagLxuUHq6/8di0fFZ1m9bevJUSPDrwo6tGLBKmIsJCP+/YPTsui6ORZVu4s4q3tR1iyMZ+/vH+Q7hGhTBqSxPQRqUzKSiI6IjhD/9DxM7y9u5gHJg9SyLczBb1IEEmMieTzY/ry+TF9OVtVzZo9R3l9SyGvbznMK5sKiA4PZcqwZG7+VB+uGZwYVH3u53MOAfCF7L4eV+I/CnqRIBUZFsq1WUlcm5XEYzcP5/29x3glt4DXNh/mpQ/ziY0O56bLUpk1ui+jM+ICuqdfXeN4Yd0hrhrYm769unldju8o6EV8ICw0hCsH9ubKgb159LPDeXtXMYs25rFw/SH+vPYAA3p3Z9aYvnxhTF+SegbeeP13dhfXLl524xCvS/ElBb2Iz4SHhnDtkCSuHZLEqbNVvJpbwAvrDjFv6Q5+8sZOrhuSxJcuz2DioMSA6IWXV1bzszd30qtbOFOHJXtdji8p6EWCyKINeRe1GFqPyDBmZ6czOzudvcWnefaDA7yQc4jXtxaSHh/NreMymZ3dl4Qe3kzOqq5x/OtzG1l/oIRf3DqKyLDgvJEc6DS8UiRInHt4yIULk13smjUVVTUs3XKYP723n7V7jxERGsK0S1O4dVwGVwyI77RevnOOH7y0lT+s2ce/3TSUe67+5Bo30nJNDa9U0IsEiQlzlze4qmNaXDTvPDS5VT9zV+FJnll7gIXrD1FaXkX/3t353Kg0bh7Zh8yE7m0tuVGl5ZX8z7Jd/Gb1Xu6+qj/fnz6sw47VVSjoRXyg/0Ov0NBvqwF7597Upp9dXlnNK5sKeC7nIGv31j6Ue2R6HNcNSeKawYmMSIttl35+8amzPL1mH79fs4+T5VXMzu7L3JmXBcS9gmCnCVMiPtCR67RHhYcya0xfZo3pSzFcwmYAAAagSURBVH5JGUs+zOfV3AJ+/MZOfvzGTuK6hTMiLZZhdcs19ImLJikmksSYSLpFhJ7X7qmoqqGsopqiU+XklZSTd7yM3LwT5Ow7xq4jpwCYNjyZb0wexKVpsW2uXZqnK3qRINFePfqLUXzqLG/vKmbNnmK25Jeyq/AUFQ08Bzc81AgLCaGyuoaqmk9mSkxkGGP69WJsv3imDktmcHJMh9TblemKXsQHvHgEYe8ekec9Dauyuoa9xac5fKKcIyfPUnTyLOWV1VRW11BZXUN4aAjdIkKJCg8lMSaS1NhoUmOj6BMXTajaM57RFb2IiA80dUUfPAthiIhIqyjoRUR8TkEvIuJzrQ56M4s0swfMbJWZ/aWR90SY2W/NbI2ZrTezKa0vVUREWqMtV/RVwHbgcWrnbDRkDlDinLsS+AzwKzPrmk88FhHxSKuD3jlX7Zx7HfjkDI6/mw48Wff+POBd4KrWHlNERC5es+PozWwy8EgDu25xzh1u5uMJQP33FABJDRzjXuBegIyMjOZKEhGRi9Bs0DvnlgPLW/nzC6kN9tK61yl12y48xnxgPtSOo2/lsUREpAEdPepmMXAPgJklA1cA73TwMUVEpJ52D3ozG2lmz9a9fAJIM7O1wEvA/c65s+19TBERaVyb17pxzq0AVtR7vRG4pe7vFcBX2noMERFpPU2YEhHxOQW9iIjPKehFRHxOQS8i4nN68IiINGnRhrxOfdiJtD8FvYg06sLHF+aVlPHwwlyAZsNeJ4jAodaNiDRq3tId5z2jFqCsspp5S3c0+blzJ4i8kjIcfz9BLNqQ14HVSmMU9CLSqPyShtcsbGz7Oa09QUjHUNCLSKP6xEVf1PZzWnuCkI6hoBeRRs2ZlkV0eOh526LDQ5kzLavJz7X2BCEdQ0EvIo2aMSqNx2eOIC0uGgPS4qJ5fOaIZm+qtvYEIR1Do25EpEkzRqVd9GiZc+/XqJvAoKAXkQ7RmhOEdAy1bkREfE5BLyLicwp6ERGfU9CLiPicgl5ExOfMOed1DecxsyJgfys/3hsobsdyOlow1RtMtUJw1RtMtUJw1RtMtULb6s10ziU2tCPggr4tzCzHOZftdR0tFUz1BlOtEFz1BlOtEFz1BlOt0HH1qnUjIuJzCnoREZ/zW9DP97qAixRM9QZTrRBc9QZTrRBc9QZTrdBB9fqqRy8iIp/ktyt6ERG5gIJeRMTnfBP0ZjbbzN43s3Vm9mOv62mKmX3ezJ4zswNe19ISdf9u3zWz1XV1d/O6psaY2bfNbI2ZrTez35lZhNc1NcfMvm9mK7yuozlm9pSZvWdmK+r+fNbrmppiZulmtsTM3jKzN8xstNc1NcTMJtb7d7rCzPaY2c/a8xi+WKbYzDKBx4BxQCnwrJnNcs4t8LayRhUB9wGbvS6kOWYWD3wbuNo5V2Zm84B7gCe8reyTzKw3EAtMcM45M3sWuBl43tvKGmdm2UB/r+tooXTgWudcsDwPcD7wHefcprr/j0Ob+4AXnHMrgUkAZhYCrATmtecx/HJFfwOwwDl3wtXeXX4SmOFxTY1yzq10zgXFbD3n3DHgqnq/3GFAQP6iO+eKnXPfqwv5HtSGfsCeTM0sGvgZ8JDXtbRQHPArM1tlZr8I8G92KUAUcJeZrQL+AzjtbVUtcgfwpnMurz1/qF+CPgE4XO91AZDkUS2+45wrN7MoM/s5EA38zuuammJmzwB7gWXAdo/Laco84GfOuSNeF9JCOcD3nXPXUPut9Pse19OUDGAU8FRdvfnAI96W1DQzCwMeAH7e3j/bL0FfyPnBnlK3TdqBmfUFXgT+5pz7R+dctdc1NcU5dxuQCYyn9gop4JjZNKCXc+4Fr2tpKefcvc65g3Uvn6e2VRqoSoCtzrkNda9fAMZ6WE9LfB5Y45wrae8f7JegfxX4nJnF1L2+C1jsYT2+YWZRwB+Ae51zr3lcTpPMbKSZ3QHgnDsD7KS23RCIpgOJZrbIzBYBl5rZ014X1Rgzizazx+rd3P40sN7LmpqxG4gysyF1r6cAG5p4fyD4GvBUR/xg30yYMrPbgAeBCmC1c+5Bj0tqlpkdds6leF1HU8xsOrX3PHbV27zcOfdDj0pqVL2e9xhq7yMcAu6uC/2AZmYrnHOTvK6jKWb2z8CdwAkgD/iac+6kt1U1zswuo7YNEgYcAe5yzp3wtqqGmVkS8CHQx3VAKPsm6EVEpGF+ad2IiEgjFPQiIj6noBcR8TkFvYiIzynoRUR8TkEvIuJzCnoREZ/7/3yQnXve4xWoAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import matplotlib.pyplot as plt\n", "from sklearn.neural_network import MLPRegressor\n", "\n", "x_max = 7. # 予測の範囲の上限\n", "\n", "# トレーニングデータ作成\n", "n_tr = 20\n", "x = np.linspace(0., np.pi*2., n_tr) # リスト 0から2πまでをn_tr等分した値を一次元配列\n", "y = make_data_by_sin_gaussian(x)\n", "\n", "# 基底関数をM次の多項式とする\n", "deg = 5\n", "X = np.vander(x, deg+1) # 計画行列の作成\n", "# X = x[:, np.newaxis]\n", " \n", "# 学習を行う\n", "mlp = MLPRegressor(activation=\"logistic\", hidden_layer_sizes=(100,), max_iter=1000)\n", "mlp.fit(X,y)\n", "\n", "# 予測\n", "x_test = np.linspace(0, x_max, 100)\n", "y_test = mlp.predict(np.vander(x_test, deg+1))\n", " \n", "#プロット\n", "plt.scatter(x, y)\n", "plt.plot(x_test, y_test)\n", "\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ランダムフォレストによる回帰\n", "\n", "\n", "訓練データを「決定木」によって分割していき、多くの分割を総合する手法をランダムフォレスト(random forest)という。https://ja.wikipedia.org/wiki/%E3%83%A9%E3%83%B3%E3%83%80%E3%83%A0%E3%83%95%E3%82%A9%E3%83%AC%E3%82%B9%E3%83%88\n", "\n", "\n", "この手法も多数の説明変数がある場合の分類(例えば画像認識)に用いることが多いが、ここでは、上のデータ、つまり多項式のそれぞれの値を独立な説明変数として入力し、ランダムフォレストのモジュールを利用してみよう。\n", "\n", "パラメータはたくさんあるが、ここでは既定値での評価だけを示す。(というか、それしかできないので。)\n", "\n", "Web検索を行うと、ボストンデータを用いた予測例などがたくさんあるので参考にしてほしい。\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD3CAYAAAAT+Z8iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3gUVdvH8e9JDwkkEBICobfQIRB6R6qiIAj2Boq9gIKgrz4FHkUpomIvYEERRVFBpUrvEGog9BZICIGQkJ7sef+YoAHSN5vZ3dyf68olO7O784uBeydnztxHaa0RQgjhvFzMDiCEEMK2pNALIYSTk0IvhBBOTgq9EEI4OSn0Qgjh5NzMDnC9qlWr6rp165odQwghHMqOHTsuaK0D89pnd4W+bt26bN++3ewYQgjhUJRSJ/PbJ0M3Qgjh5KTQCyGEk5NCL4QQTk4KvRBCODkp9EII4eSsKvRKqTuUUguUUqfy2f+sUmqbUmqXUupFa44lhBCiZKw9o48DngQ8rt+hlOoK3A10BToAQ5VS4VYeTwghRDFZNY9ea70GQCmV1+7BwBytdUbOc74AhgAySd4W4o9C1O9QuR4EtwD/OpD3z0UIUc7Y8oapAGBTrsfngI55PVEpNQYYA1C7dm0bRnJC6Umwdjpseh8smf9s96wE1VoYRf/qf4Oagbu3eVmFEKawZaGPBYJyPQ7O2XYDrfUnwCcA4eHhshJKUWgNe3+A5a9B0jlocy/0nADJFyBmL8Tug5h9sOs7yEgyXqNcIKDhP4U/uJXx54rBcvYvhBOzZaH/BZiplPoSsAAPAk5zQXZRRDTTlkZxNiGVGv7ejB8QytCwkLI5+Lnd8PsEOL0ZaoTByK+hVntjX+W6UDPXpRCLBRJOXlv8o7fD/p/+eU6FAKPg1+oA3caBR4Wy+T6EEGWi1Au9Umo1cJfWertS6ldgC5ANzNdaO8X4/KKIaCb9tJfUzGwAohNSmfTTXgDbFvvkeFg1GXbMNYrzbbONM3mXAq6pu7hAlXrGV7Pb/tmedhli9xuFP3av8UGwdjqc2AD3zAcvP9t9H0KIMqXsbc3Y8PBwbe9NzbpOXUV0QuoN20P8vdkwsU/pHzA7C3bMgVVTjDH5jo9Bz5fA2790j7P/Z1j4KAQ1gft+Bt88G+EJIeyQUmqH1jrPmY12173SEZzNo8gXtN0qJ9bDHy8Zwy71esCgtyCoaekfB6D57eBREb6/D+YMhPsXgX8t2xxLCFFm5M7YEqjhn/fMlfy2l8jlM/DDwzD3FmOYZeRX8MCvtivyVzXqCw8sgitx8MUAiDtk2+MJIWxOCn0JjB8Qire76zXbvN1dGT8g1Po3z0yDtdNgdntjXnzPifDUVmg2pOxmxtTuBA8vgexM48z+bETZHFcIYRNS6EtgaFgIbwxrSYi/NwpjbP6NYS2tuxCrNRz8HT7oaIzFN+xrFPjek8yZBRPcEkb9Ce4+MPdWYwhJCOGQ5GKsvVg73ZhRUzUUBr0JDXqbnciQeBa+GmpM0RzxJYQONDuRECIPBV2MlTN6e3BkpXEW32I4PLHBfoo8QKUa8PAfxrWB+ffAngVmJxJCFJMUerNdOgkLRxuF9Lb3wNXd7EQ38gkwLgTX6QI/jYGtn5qdSAhRDFLozZSZBgseAEs23PkNePiYnSh/XpXg3h8hdBD8/iKsmWZcVxBC2D2ZR2+mP8bDuV1w17cQ0MDsNHm6vtXDhH5TGeJZCf6aAmkJ0H+K9MkRws5JoTfLji9h51fQ/QVocovZafKUV6uHiYsOoG//P4Z6+cGm2ZCaALe+A67yV0kIeyVDN2Y4GwG/j4f6vaD3K2anyde0pVF/F/mrUjOzmbbssDEzqNck2PUN/PgQZKWbE1IIUSgp9GUt5SJ8/wD4BsHwL8DFtfDXmKTAVg9KQa+JMHAqHPgN5o2A9CtlnFAIURRS6MuSJRsWPgJXYmDkl8ZsFjtWpFYPnZ6AoR8aN1R9NQQykssonRCiqKTQl6XVU+HoSrh5GoS0MztNoYrc6qHNPTBirtHn/q/Xyy6gEKJIpNCXlag/Ye1bEHYftH3Q7DRFUqxWD81ug3YPw+YPIHpHmWcVQuRPWiCUhfij8ElvqFwHRi9z3nVb0y7D+x2NRVHGrLbPm7+EcFLSAsFMGSnGTVFKwZ1fO2+RB2NVqltmGL3zN8wyO40QIodMfrYlrWHxWGPJvnt/MNZzdXZNboFmQ8le/Sb3ra/G5sSAsl9TVwhxDTmjLyXnk9L4OeIMFkuuobDtn8Oe+cZ880b9zAtXxv6oNY6kbA/Gps0GLH+vqbsoItrsaEKUS3JGXwqupGfxwOdbORiTRFxSOmN6NIDT2+CPidCoP/QYb3bEUqW1JjYxnYMxiUTFJBEVk8TBmCQup2Yyrl9jZqy+QJes+5ju/jH3uq7km+x+xo1WS6PkrF4IE0iht1K2RfPsdxEcPn+FNrX8efPPKDoGWWi95AHwC4Fhn4CL4/7idCU96+9iHhWTyIGYJA7FJpGQkvn3c4IreREaXBF3Nxde+GE3AD/Sg9tcNvKS23xWZLclhgDbrKkrhCiUFHorTVkSyaqD55kytAW3tq7BkHdXk/X9Q2jXi6jRy8G7stkRi0RrzdG4K0SeMwr61bP0M5f+Kc6+nm40rubLoBbVaRJckdDgijQJroh/BQ/A+ND7fP0xXv/9IKB4OWs0yzxeYor7FzyS+SI1/E1YKUsIIYXeGl9vOsGcDScY3a0e93WqA8CChisI2ruPz/0nMCq4JaXZ1/H6TpKlcYEzM9vC4j1n+XTtcSLPJQLg6qKoX9WHNrX8ubtDbUKrGUW9ZmVvVAGdKl1dFGN6NCA7W/PW0ijO6CBmZN3Bq+7zGK620H3A41ZlFUKUjBT6ElpzKI5//xZJ36ZBvHxzU2Nj5K8E7f2IAzVHMPlIG/T64zzSvX6pHC+vTpKTftoLUKJin5iWyXdbTjF34wnOXU6jYZAvk4e2oF3tyjQI8sHTreQ9eJ7o3ZDAip7857dI5qQP4jbXzfzX42t8QseW+D2FECUnN0yVQFRMEnd8uJFaVSrww+Od8fF0M26K+rgnBIaiH1rCY9/tY9XB8yx4vDNta1s/fNN16iqi8xjjDvH3ZsPEPkV+nzOXUpiz4QTzt54iOSObzvUDGNOjPj0bB+LiUvp95XedTmD2t4v4MGUckQH9aPz4t3h72G8jNyEcldwwVYriktIZNXcb3h6ufP5QuFHkAf54CZQLjPwS5e7FtDtaE+znxTPfRpCQkmH1cQvsJFkEu08n8PS3O+k5bTVfbjxBv2bVWPxMN74b04neTYJsUuQB2tTy572x97Ox+v20vvgn/5r5DrtOJ+T7/EUR0XSduop6E5fQdeoqmZIpyo8TG4zGhzYghb4Y0jKzGfP1di4mZ/D5g+2p7pdzl+vRVXBkOfQcD341AfCr4M7se9pyPimNF3/Yg7W/ORWpk+R1LBbN8shYRn68iSHvb2BNVByju9Vj7YTezLorjBYhflZlKipvD1d6PvIWKZUaMDbtA+7/cBUzlkWRkWW55nlXh6eiE1LRIPPvRfkRsxfm3gIb37XJ20uhLyKLRfPiD7vZdTqBt+9sQ8uaOUXSkg3LXgX/2tBhzDWvaVPLn4mDmrLiQCyfrz9u1fGL3EkS4wNp3paT9J25hke/2k70pVT+75ambJzUh5dvblrgh4PNuHlSYfj7VCeO94OX8N6qI9z+wQaiYpL+fkq+C50sjSrrtEKUreX/MlqItHvIJm8vF2OLaNaKQyzec45Jg5owsEXwPzt2zzd6u9zxBbh53vC6UV3rsvlYPG/+eZDwulVoU8u/RMe/esG1oFk3F66k89Wmk3yz+SQXkzNoVdOPd+8O4+YWwbi52sFnep3O0P4Remz7nPk3j+CpNWnc+t56XhzQmNHd6ls9PCWEQzq6ymhf3v9/NpuOXe4vxhZlyuJPO88wbsFu7gyvxdThLf+ZYpiRAu+1hUoh8MiKfBfJvpySyc3vrgPg92e741ehdLs6ZmZb+HD1UWb/dYSMLAt9mwbxaPf6dKhXpcDpkKZIS4QPOoFnJS7ct5xXfo1i6f5YOtStwsn4ZGKTblySsLgXnIVwGBYLfNLD6Pz69PY8TxaLSi7G5qMoY8Jbj19k4sK9dGkQwOShLa4tnJveh6Rz0H9KvkUero7XhxGbmMaLP+62erw+t0OxSQz7YCMzlx+if7NqrHyhJ5892J6O9QPsr8gDeFWCwW9D3AGqRnzAR/e1Y8aI1hw4l0hCaiaebtf+lcxveEoIp7B3gTE+3+c1q4p8Ycp1oS9sTPjEhWQe+3o7Nat48+G97fDIXYSSYmH929D0VmNIohBhtSszcVATlkfGMmfDCauzZ1s0H605yuB31xOdkMqH97Zl9j1taRDoa/V721zjAdDiDlg7DRUXxfB2NfntmW5kWzSd6gcUbaETIRxdZhqsnAzV20CL4TY9VLkeoy9oTPhySiajvtwGwBcPtr9xuGX1G5CdDn3/U+Tjje5Wj83HLvLGHwdoW6dyicfrj8Vd4YUfdhNxKoGBzYOZcnsLqvra7mzAJgbmLKv46zMw6k/qVvVhcKvqrDhwnk2T+lDRSxYtEU5uy0eQeAZu/9Dm/bDK9Rl9frNPqvt58cS8HZy+mMLH94dTt6rPtU84fwB2fgntH4GABkU+nlKK6SNaEVTRi6e/3cnl1MzCX5SLxaL5fP1xBr2zjmNxybxzVxs+vK+t4xV5AN9Ao9if2QrbPgPg4a71uJKexQ/bz5gcTggbS7kI62ZCowFQr4fND1euC31eUxa93FyoXaUCG4/G8+bwVnSoV+XGFy5/DTwqQo8JxT6mfwUP3rsnjJjLaUwoxnj9qfgU7vp0M5MXR9K1YVWWje3BkDYh9jkOX1St7oQGN8GK/0DCKVrX8qddncrM3XiCbIt9TRIQolStnQYZSdD332VyuHJd6PNa/Lp/82A2H7/IM30aMqxtzRtfdPQvOLwMerwAPgElOm7b2pV5aWATlu6PZe7GEwU+V2vN15tPMvCdtRw4m8hbd7Ti8wfDqVbJq0THtitKwa05Sw4uHgdaM6prPU5dTGHVwfPmZhPCVi4eh62fQpt7oVqzMjlkuR6jB6PYX73Y9+e+GJ6Yt4PBraoztm/jG5989eYov9rQ4TGrjvtI93psOR7P678foF2dyrSqeeN4fXRCKi/9uIf1Ry7QvVFV3hzeypybnWzJvzbc9Br8+RLs/YEBze+ghp8XX6w/Tr9m1cxOJ0TpWzUZXNyg9ytldshyfUaf277oyzz/fQRtavkzfUTrvHu/7PkeYvdC33+Bu3Vn1MZ4fWsCfT156rrxeq01C7adZuDba9l56hL/u70FX43qUOIib/f9Yzo8CjXbwx8v4ZaewINd6rLpWDyRZxPNTiZE6YreAfsWQpenoVL1MjusVYVeKTVSKbVVKbVDKTUjj/2rr/vqYM3xbCU1I5vn5kfg7+3BJ/eH4+WeR3fFjBRjKlRIu1KbCmWM17flXEIaL/24J2eJvjRGzd3GhIV7aFajEkuf78G9HeuUeCzeIfrHuLjC4FmQegk2vsdd7Wvj7e7KnA3WtY0Qwq5oDctegwpVocuzZXroEhd6pVQdYDLQDwgHaiqlrq+AnlrrXrm+tlqR1Wbe/PMgR+OSmTGyNYEV85nBsvl9SDpb6M1RxdWuTmUmDAzlz/0xvPjDHvq/vZZNx+L5163N+O7RTtSqYt2qTA7TPya4BbQYBls+xk9f5o52Nfll11kuXLnxTlkhHNKhP+Hkeug10bhxsAxZc0Y/EFiotb6sjakjHwNDr+5USrkBfkqpBUqptUqpyUopu2tEvvZQHHM3nuDhrnXp2rBq3k+6ch7Wz4Img6FOl1LP8Ei3+vRpEsTCnWdoEOjD78925+Gu9UqldbBD9Y/pORGyUmHDLB7qWpeMbAvzNp8yO5UQ1svOMhqXBTS0WeOyglhT6AOAmFyPzwFBuR77AmuAMUAvoDrwSF5vpJQao5TarpTaHhcXZ0Wk4klIyWD8j7tpGOTLSwOb5P/E1W9AVlqxbo4qDhcXxbt3h/HpA+H88HgX6pfi3a0laW9smsDG0HIkbP2MBl7J9AoN5OvNJ0nPsk2PbiHKzK5v4EKUMZ3StexvBrSm0MdybWEPztkGgNY6QWv9RM5/LcBPQJ5j9FrrT7TW4Vrr8MDAQCsiFc//LdpH/JUMZt3ZJu9xeYDzB2HHlxA+Gqo2tFkWX083+jWrhmspLwBSnPbGdqHnBMjOgA2zGNW1HheupLN49zmzUwlRculX4K/XoVZHY1TABNYU+t+B25VSFXMejwJ+ubpTKRWslHpZ/XMVcSCw04rjlapfdkWzeM85xvZrXPACHCv+BR4+0POlsgtXivK6V8Cu+8cENIDWd8O2z+kenEnDIF++2HC8VBvBCVGmNr0PV2Kh3+RSvb5XHCWeR6+1PqeUeh1Yq5TKANZprRcqpVYDd2Gc3fsCO5VSV4BdwCelkNlqZxNSeXXRPtrW9uexHgUs3n1sjXEBpe9/SnxzlD3Ifa+AQ+jxIuyZj1r/NqO6PsPLP+9l24lLed+lLIQ9u3IeNrwDTW+D2h1Ni2HV9Eqt9TytdZjWuqPW+sWcbb201jHa8HLO/u5a62e01sVr7mIDFotm/I+7ybJoZo5sk/+CHBYLLHsF/GpBx8fLNmR5V6WecdfgjrkMa6Dxr+DOF1au0CWEKf5ufvhvU2OUuxum5m48wYYj8bw6uNmNzcpy2/O90Sf6JutvjhIl0ONF0BqvTW9zT4faLIuM4fTFFLNTCVF0cYdyru+NKlbzQ1soV4X+cGwSU/88yE1Ngrirfa38n5iRYtymXCPM5n2iRT78a0O7ByHiax5qplBK8WUhfYGEsCsr/g3uFUrU/LC0lZtCn5FlYeyCXfh6ujF1eKuC7zTd/AEkRhs3R9m4T7QoQLdxoFwJiniPm1tW5/ttp7mSnmV2KiEKd3IjRC2Bbs8ZLblNVm6q2LsrD7MvOpE3hrXM/+5XyLk56m0IvQXqdiu7gOJGfiEQ/jDs+pbHWyqS0rP4cftps1MJUTCtjeaHFatDp6fMTgOUk0K/4+RFPlh9hBHtajKgeXDBT149FTJToZ9tbo4SxdRtLLi60/zwx4TV9mfuxhNYpFe9sGeRiyB6u9Gd0sO6FialxekLfXJ6FmO/300Nf29eu7WQ3s9xUbBjrnHxpGqjMsknClEx2FjJa898nmkFJ+JT+CtKetULO5WVYSykE9QM2txjdpq/OX2hn7IkktOXUpg5sk3h65Auz7k5qtdEm+Wx+5bB9qjrc+DmRa+YOVT38+IL6Wop7NWOOXDpOPT7r9GV1U44daFfeSCW77ae5rEeDQq/2eb4Ojj0B3QfBz75NDezkkO0DLZHvkHQ4VFc9v3Is62y2XAknoMx0qte2Jm0y8bQb70e0LCv2Wmu4bSFPv5KOi8t3EPT6pUY268IwzBr3jQuntjw5iiHaRlsj7o8Bx4+DE/8Bi93F+asP2F2IiGutX4WpF40tdVBfpyy0GutmfTTXhJTs5h1Zxs83Qr5FerUFjixDro8A+626+roUC2D7Y1PAHR8HI+oX3iiaTo/74omXnrVC3uRFGNMy245Emq0MTvNDZyy0P+w4wzLImMZPyCU0OCKhb9g3XSoEGDzPtEO1TLYHnV+CjwrMTpzPhlZFr7dIr3qhZ3Y+J7RdbX3JLOT5MnpCv3piyn897dIOtWvwuhu9Qp/wbndcHgZdHrCuBBrQw7XMtjeVKgCnZ7E9/gf3F/3Ml9vPklGlsXsVKK8S7kI2+dAizugSgFNEk3kVIU+26J5YcFuFOS/wPf11s0Az0rQ/lGb53O4lsH2qNMT4OXHc64/cj4pnd/3Sq96YbItH0FmsjGRw06VuE2xPfp03TG2nrjIzJGtqVm5CDcqxEVB5K/GD8jb3/YBccCWwfbG2x86P0PVv6YwqMogvtjgx5A2NUq8eLoQVklLNAp9k8EQ1NTsNPlymjP6yLOJzFgWxaAWwdxe1EK6/m3j4munJ20bTpSujo+Bd2Ve8fmFPWcus+PkJbMTifJq+xfGtMruL5idpEBOU+i3nbhIgI8n/7u9ZdHO7i6dgD0LjAuwNpo3L2zEqxJ0fY6acevo5nVMbqAS5shMNVaPatAHQtqanaZATlPoH+xSlxUv9KSKj0fRXrDhHePOtS7P2DaYsI32j0KFqkyu9Ct/7ovhzCXpVS/KWMQ3kHze7s/mwYkKPRgLbBdJ4jnjh9TmHqhUw7ahhG14+kK356mXuJX2LlF8temk2YlEeZKdaZws1uoEdbqanaZQTlXoi2zje2DJhq7Pm51EWCN8NPgE8d9KvzJ/6ymSpVe9KCt7vofLp42zeQeYCFD+Cn1yvNF4qOUdxtqkwnF5VIDu4whNjaB5xm5+2nnG7ESiPLBkGxM5gltCo35mpymS8lfoN38AmSnG6kXC8bV7CF2xOv9XYRFz1h+XXvXC9iJ/gfgjDnM2D+Wt0Kddhq2fQtNbIaiJ2WlEaXD3RnV/geZZ+6l+aQuLdkknUGFDWsO6mRDQCJreZnaaIitfhX7rp5Bu/3NeRTG1fQBdqSavVfiZ/y2O5HJKptmJhLM6vAxi9xo3WdpRv/nClJ9Cn5FsDNs07As1wsxOI0qTmyeqx4uEZh2kedoO3lp60OxEwhlpDWung19taDnC7DTFUn4K/Y4vISUeur9odhJhC23ugYrV+W+VZXy79RQRp+RuWVHKTqyHM1uh67PgWshqdXamfBT6rHTY+K4x37VOZ7PTCFtw84TOT1P3yk76+JzklZ/3kZUtnS1FKVo3HXyCIOw+s5MUW/ko9Lu+haRzMjbv7No9BN6VeT1wOZHnEuUmKlF6zuyAY6uhy9M2XZzIVpy/0GdnwYZZxrh8gz5mpxG25OkLHR6j2rlV3FP3CjOWRRFzOc3sVMIZrJsBXv4QPsrsJCXi/IV+30KjgVn3Fx1mzquwQsfHwN2Hl/2WkmXRTF4caXYi4ehiIyFqibGetGcRVqyzQ85d6C0WWD8TgppB6M1mpxFloUIVaPcQvocW8XJnb5bsPcfqqPNmpxKObP1McPcxTiIclHMX+oOLIe6gMTbv4tzfqsil81OgXLjP8iv1A3147Zf9pGVmm51KOKL4o8aoQPtRxkmEg3Le6qe1cZW8Sn1ofrvZaURZ8guB1nfhunseb/YP5tTFFD7464jZqYQj2vAOuLhD56fNTmIV5y30R1YaC393G+tQd7CJUtL1echKp33sfIa2qcGHa45yNO6K2amEI7kcbczYa3s/VAw2O41VnLfQr5sOlWpCq7vMTiLKyKKIaLpOXUW9iUvo+tkpomv0h22f8399a+Ll7sqri/ahtTQ9E0W0aTZoC3R51uwkVnPOQn9iA5zaZNzB5lbEFaeEQ1sUEc2kn/YSnZCKBqITUnnmdG9IT6Tqga+ZMLAJG4/G8+vus2ZHFY4g+QJsnwOt7oTKdcxOYzXnLPTrpoNPILR9wOwkooxMWxpF6nUXXHdm1mazSxhs/oB7wgJpXdOPyYsjuZwqTc9EITZ/CFlpxtCvE3C+Qh+9A46uMmZeOOAdbKJkziak5rl9VupgSI7Ddfc8/nd7Sy4mZzB9aVQZpxP2LvewX/83fiNz08fQ7DYIbGx2tFLhFIU+9w9pzeeTyHCvZCwzJ8qNGv55f6ifrhQGNTvAxndpEezDA53r8s2Wk+w+nVDGCYW9un7Yr++VX3HPSuKvwPvNjlZqHL7Q5/4hNVKn6WnZwqfp/Vh0IMnsaKIMjR8Qirf7tbOrvN1dGT+wifHrd8Ip2LeQF/o3JtDXk1cW7SVbVqMSXDvs50U6o93+4K/s1vzfFueZrWdVoVdKjVRKbVVK7VBKzchj/7NKqW1KqV1KKZv0B879Q3rS7ReStSefZvRnmvx6Xq4MDQvhjWEtCfH3RgEh/t68MawlQ8NCoPFACGwK69+moocrrw5uxr7oRL7edMLk1MIe5B72u9t1FQEqidlZQ/MdDnREbiV9oVKqDjAZ6AAkAvOVUsO11gtz9ncF7ga65rxklVJqtdZ6u5WZr3H1h1FHxXCryyY+y76ZBCpy2Yl+SKJohoaFGIX9ei4uxln9z2Pg0J8MbjWIBdtPM33ZIQa1rE61Sl5lH1bYjRr+3kQnpOJOFmPclrDF0oQdOpSQfIYDHZE1Z/QDgYVa68vamJz8MTA01/7BwBytdYbWOgP4AhiS1xsppcYopbYrpbbHxcUVK8TVsdlhruvJwo3Psm6+ZrsQALQYDv61Yf1MFPDfIS3IyLYweXHktfPvp65iUYSsO1ueXB32G+a6jurqIrOzhhrDfgNCzY5Waqwp9AFATK7H54CgYuz/m9b6E611uNY6PDAwsFghrv6QZmUN47aMycRR2el+SKIUuLoZN76c2QYn1lOvqg9P9mrA4j3nmPDjnmvm30/6aa8U+3JkaFgIU4c25RmP39hjqcexih3+GfZzEtYU+liuLdzBOduKur9UXB2breHvwyFd+9qxWSFyC7vPuL9i/UwAHu/ZAFcXRcZ1K1GlZmbLNZ5yZoj7VmrqGFrdPZkNk25yuvpR4jF64HdghVLqTa11EjAKWJRr/y/ATKXUl4AFeBCwyQXZfMdmhcjN3Rs6PQkr/wNnI/CqEZbvzBtnuhAnCmGxGAuLBDaF0FvMTmMTJT6j11qfA14H1iqltgCxWuuFSqnVSqngnIuuvwJbgM3Ab6V9IVaIYms/Gjwrwfq3AfK94CbXeMqRqCUQd8Cp25lb9V1predprcO01h211i/mbOultY7J+fN0rXVbrXV7rfUN0y+FKHNeftD+EYj8FS4cZvyAULzcrv1nINd4yhGtYe10qFzPqduZO+fHlxAF6fQkuHnChlnGhbjhrfDzdgegqq+HXOMpT46shHO7oPs444K9k5JCL8of30AIux92fw+XoxkaFsLGiX3wcHPh1tY1pMiXF1rD2mnlop25FHpRPnV5xug1vmk2AD6ebnRvWJVl+2OlZ315cXIDnN4MXZ9z+nbmUuhF+VS5DrQcATvmQspFAAY0DyY6IZX9ZxPNzSbKxtrp4BNkrCDl5Jx3UEqIwnR7HvbMhy0fQ/5wZTkAABxRSURBVO9J3NQ0CBcFyyJjaRHiZ3Y6YUtndsCxv6Dff/9uZ56cnkVyepapsXy93KjgUfplWQq9KL+CcuZNb/kIujxDgK8v4XWrsGx/DOP6OUcfcpGPddPByx/CRwGw50wCd32ymZSM7EJeaFtThrbgvk6lv6KVFHpRvnUba8yj3jEXujzNgObBTF4cycn4ZOoE+JidTthCzD6I+h16vQyeFUnLzGbcgt34ebvz8s1NUcq8aB3qVrHJ+0qhF+VbrfZQt7txUbbDo/RvVo3JiyNZtj+WR3vUNzudsIV1M8CjInQcA8CMZVEcOX+Fr0Z1oEfj4vXachRyMVaIbmMh6Rzsnk+tKhVoVr0SyyJjCn+dcDwXDsP+n6HDI+Bdma3HL/LZ+uPc16m20xZ5kEIvBDToA9Vbw4Z3wJJN/+bV2H7yEnFJ6WYnE6Vt/dvg5gWdniI5PYsXf9hNrcoVmDSoqdnJbEoKvRBKGX1OLh6FfT8xoHkwWsPKA6XebFWY6dJJ2PM9tHsIfAN5448DnL6UwvQRrfHxdO5RbCn0QgA0uRWCmsGaN2kSVIFaVbxZul+GbxxNgYvIbHgHUNDlGdYeiuObzad4pFs9OtSzzQVQeyKFXggwuhb2mgjxh1H7FjKgWTAbjsRzxeR51aLoFkVEM+mnvXkvIpN4DiK+gbB7uewRxEsL99AwyJcX+peP5nVS6IW4qsmtUK0FrHmT/k2rkpFtYXXUebNTiSKatjSK1Mxr58H/vYjMptlgyYKuz/Pf3yI5n5TOjBGt8XJ3NSlt2ZJCL8RVV8/qLx4lPHEFAT4eLNsv4/SOIr/FYlITYmH7F9ByBMvOebNw5xme6tWA1rX8yziheaTQC5Fbk8EQ3BKXddPo3ySAvw6eJyPLUvjrhOnyWyzmWZ8VkJlKQvgzvPzzXppVr8TTfRqVcTpzSaEXIjeloNckuHiM+322kJSexaZj8WanEkUwfkAo3tcNxQS5p3Gv+hPd7DZeXpfO5dRMZt7ZGg+38lX6ytd3K0RRhN4M1VvT5NBH+HlomX3jIIaGhfDGsJaE+HujMJaJnNt8F+5ZV1hT7QF+3xvD2H6NaRJcyeyoZc65J48KURI5Z/Uu393F+OBdvBPpxZQhLXBxMbEJiiiSoWEh/ywck5EMb48mvV5fnlttIay2P2O6l8+2FnJGL0ReGg+EGmHcfuVbLiUlE3E6wexEori2z4HUi0xLvZX0rGxmjGiNm2v5LHnl87sWojBKQa+X8UmJ5k63tdL7xtFkpsHG94gN6MhnJwKZOLAJ9QN9zU5lGin0QuSnUT8ICWes56+s2ndGlhh0JLu+gSsxTIrrT+f6ATzQua7ZiUwlhV6I/OSM1VfNPk/7hD84cv6K2YlEUWRnotfP4pB7U7aqFrx1R6tyf31FCr0QBWl4ExnV2/GU2yKW7z1ldhpRFHsWoC6f5o3kwbw6uBm1qlQwO5HppNALURCl8LjpFUJUPCriG7PTiMJYsslYM51IXRca9mNkeC2zE9kFKfRCFKZBH85VasOQK/M5e0Fm39gzy/5FeCQc43M1jKl3tEaZuS6gHZFCL0RhlEL3mkgNdZFTKz4yO43Ij8XCpT9f57AlhB5DRlGtkpfZieyGFHohiqBG2ED2uDaj8aFPjKl7wu6c2foTAclH2FD9fm5rU9PsOHZFCr0QRaEUkaFPUcUST8rmz8xOI66TmZVN8vI3OUMQt977rAzZXEcKvRBF1LTzYDZbmqLWvw2ZebfEFebYtuxbQrMPcSnsKQIq+Zgdx+5IoReiiFrV9ONLj3vwTr9g3F4v7ILOyqDW9jc47RJCi1ueNDuOXZJCL0QRKaUIbNmHzbo5ev3bkJFidiQBHP7jXWpZojkV/jLKzcPsOHZJCr0QxTCgeTAzMoajks8bqxYJc6VeovrOWWxTrWjf726z09gtKfRCFEOHelU45NWSQz7hsGGW0QpXmOb8kin4WK5wuv3LeJST9V9LQgq9EMXg7urCTU2C+F/KUEiOg22fmx2p/Io/SpV9c/lZ9ab/Tf3MTmPXpNALUUz9m1djTWp9LlXvDhvegXRpdmaG5N9fJV27EtP2BXw9ZQ2lgkihF6KYejQOxNPNhYWV7oeUC7DtU7MjlT8nN+JzdAmfWIYwone42WnsnhR6IYqpgocb3RsF8sWJquiGfWHDu5CeZHas8sNiIfP3SZzTVbjU+lGCKkqrg8KUuNArwxtKqS1KqV1KqXvzeI6bUuqCUmp1ri+Z/yQc3oDm1Th7OY2jzZ6F1Iuw9ROzI5Ufe3/APXYX07Lu5MGezcxO4xCsOaO/B2gEdAJ6AK8opapf95xawDKtda9cXxlWHFMIu3BT02q4KPjlQjA0GgAb34O0RLNjOb+MFCwr/s1+6pMSOowG5Xh5wOKwptAPBj7RhkTgR+Dm655TFwhSSv2hlFqnlLrLiuMJYTeq+HjQoV4Vlu6PgV4TIfUSbP3Y7FjOb9P7uCSd5T/p9zKmVyOz0ziMQgu9UqrPdUMvq5VSq4EAIPeKyeeAoOtengKsxvhQGAyMV0rd8LuWUmqMUmq7Ump7XFxcCb8VIcpW/2bBHIq9wnHPUGg8CDbOhrTLZsdyXkkx6PVvs9qlE9TpStvalc1O5DAKLfRa61XXDb300lr3AmK5trAH52zL/dotWuspWutsrfVlYCXQLo9jfKK1DtdahwcGBlr1DQlRVvo3rwbAsv0x0PtlSE+ElZNNTuXEVk1BZ6Xzr9SRPNazvtlpHIo1Qze/AKMBlFIVgGHAH7mfoJTqenW4RinlCfQCIqw4phB2o2blCjSvUYllkbFQvRV0eAy2fQant5odzfnE7EVHfMPPHoPxCGxI79DrBw9EQawp9AuBs0qp7cAaYKrW+pxSqo1San7Ocw4Aw5RS2zCGcD7RWu+zKrEQdmRA82B2nrrE+aQ06PMKVAqB356D7EyzozkPrWHpK2R5+PGfyzczpkd9XFyk33xxlLjQ51yEfSFnyKW91npezvZdWuu7cv58UWs9Mmd/Z621rNggnEr/5tXQGlZEngfPinDLdDgfCRvfNTua8zi8DI6vYZ73PXhXCmBImxCzEzkcuWFKCCuEVqtInYAKxuwbgNBB0GwIrH4T4o+aG84ZZGfCsv8j3a8+U2I7MbpbPTzcpGwVl/wfE8IKSin6N6vGxqMXSErLGa4Z9Ba4ecHi541hB1FyO+bChUPM8RmFt6cXd3eobXYihySFXggrDWgeTGa25q+onKnBFYOh37/h+FrY/Z2p2RxaagL89TppNbvy1vF63NupDhW93M1O5ZCk0AthpbDalanq62FMs7yq7UNQqxMsfRmSL5iWzaGtmw6pl/iiwiO4ubjycNe6ZidyWFLohbCSq4uiX7NqrI6KIy0z29jo4gK35rQwXvqKuQEd0cVjsOVj0lrcxTuR3tweFkK1StK8rKSk0AtRCga3qsGV9CyGfbCRbScuGhuDmkD3cbBnPhxdZW5AR7Pi3+Dixpde95OeZeHRHnKDlDWk0AtRCro2rMqH97YlISWDER9tYuz3uzifmAbdxkFAQ1g8VhYTL6pTmyHyFzI6PcuHO5Pp16waDYOkeZk1pNALUUoGtazOihd68kyfhizZc44+M9bw6aazZN0yCy6dgDVvmh3R/lksxnWNijX43n0ICSmZPN6zgdmpHJ4UeiFKaFFENF2nrqLexCV0nbqKRRHRVPBw44X+oSwb24MO9arwv98PMHCRhdgGI41WxjF7zY5t3/YthOgdZPV+lY82xtC+bmXa1ZHmZdaSQi9ECSyKiGbST3uJTkhFA9EJqUz6aS+LIqIBqFvVhy8eas/nD4aTkWWh//6+JLpUIuPnp8GSneeHRLmXmWqMzVdvzRLVneiEVB7rIWfzpUEKvRAlMG1pFKlXZ9jkSM3MZtrSqGu23dS0GsvG9uCRfm35d+b9eMTuYsGHrzFx4Z58PyTKrU3vQ+IZdP8pfLT2BA2DfOnTRJqXlQYp9EKUwNmE1CJv93J35ZmbGjHu+Ynsq9CBm89/RuWsa9ddyOtDorTZ9W8RCadg/dvQZDDrMpty4FyiNC8rRVLohSiBGv7exdoOULOKDy0e/QwXNP91nwNc2x4hvw+P0lDYUJOpMlJg/r2gXKD/FD5eexQ/b3dmLT9knx9KDkgKvRAlMH5AKN7urtds83Z3ZfyA0IJfWLkOn7vfTT/XnQx02XbNrmA/290QVNShpjKntdHWOWYvDP+MvSlV2HAknuT0LM5eTrO/DyUHJYVeiBIYGhbCG8NaEuLvjQJC/L15Y1hLhoYV3kK39qBxROq6/Md9LpVI/nt7WmY2W47F2yRvcYaaytTmD2DvAqOXf+MBfLz2KArIslz7245dfCg5MDezAwjhqIaGhRSpsF9vSLs6/JU4jdA1dzLBbT4f+j7NsLAQFu89xz2fbWH8gFDGdC/d8eka/t5E51HUCxpqsrljq2HZ/0HTW6H7i5yKT+H3vefIr9+n6R9KDkzO6IUwQe/e/XHt/CT3ua1kwz0VeGFAKL8+3ZWBzYOZ+sdBxny9ncsppbdKVYmHmmzl0gn44WGoGgpDPwSl+Gz9MVxdFMH59LQx9UPJwUmhF8IsvSaBX21jjDorg4pe7sy+J4x/3dqMNYfiuOW9dew9c7lUDmXNUFOpy0iB+feBzoa75oFnReKvpLNg+2luDwth4qAm9vWh5ARk6EYIs3j6wi0z4NsRsOEd6DkepRQPd61Hm1r+PDVvJ8M/3Mirtzbjvo61Ucq6oZySDjWVKq3h16chdh/c+yMEGDdEfbTmKBlZFsb0aPB3X5tpS6M4m5BKDX9vxg8INT+7A5NCL4SZGveH5sNg7VvQfChUbQQYPe6XPNud57/fxauL9rH9xEVev70lPp4O/k9243tGm4Ob/gWN+gJwPjGNrzadZGhYyN9F3i4+lJyIDN0IYbaBU8HdG367dunByj4ezHmoPS/2b8xvu88y5P0NHI5NMjGolY6ughX/gmZDodvYvze//9cRsiya525qZGI45yaFXgizVawG/SbDyfXw58Rrir2Li+LpPo34ZnRHElIyuG32Bn6OOGNi2BK6eNy4+BrYFIa8DznDUNEJqXy39TQjw2tSJ8DH5JDOSwq9EPag7QPQ6SnY8pHRu95iuWZ3l4ZVWfJsd1qG+DH2+928/PPef1azsncZycadr5Bz8fWf3vKzVx0B4Ok+cjZvS1LohbAHSsGA/xlDGjvmwC9PgeXaQl6tkhffPtqRx3s24Nstp7jjo42cirfzxUy0Nr6XuANwxxdQpd7fu07Fp/DD9tPc3aEWITJ10qak0AthL5QyLlL2ehl2fws/PQrZ186ld3N1YeKgJnz6QDin4lO45b111y5Kbm82zIL9P0Pff0PDm67Z9c7Kw7i6KJ7q3dCUaOWJg1/CF8LJKAW9XgI3D6M3e3YGDP/CeJxLv2bVWPJsd56ct5MxX+9gTI/6jB8QirvrteduFosmITWT+CvpXLiSwcXkDOKTr/45nfgrGcZXcjoWDdNHtKJdnSql870cXgEr/gMthkOXZ6/ZdeT8FX6OOMOorvUIkkW/bU5pnd8Nx+YIDw/X27dvNzuGEObb/KFxcbbRABj5FbjfWBDTMrOZsiSSbzafonVNP2pVqUB8roJ+MTkDSz7/xCtXcCfA15MqPh5U9fUg4lQCbq6KP57rga+10zjjj8KnvcGvFoxeBh7XXmh95rsIVh6IZe2E3lT19bTuWAIApdQOrXV4XvvkjF4Ie9XpCXD1gCXjYP7dcOc88KhwzVO83F2ZMrQl7etW4a0/o0hMyyLAx4M6ARVoW6cyVX09qOLjQYCvJwE+HgT4ehDg40nlCu64XXf2v+3ERUZ+vIn/LTnAG8Naljx3+pV/2g7fNe+GIn8wJpHFe87yRM8GUuTLiBR6IexZ+9Hg5gm/PA3fjuS35jOZuvLMDXeMDmljfFl1qLpVGNOjPh+vOUb/ZtXoXZLVnbSGRU/AhSi47yeoXPeGp7y9/BC+Hm6M6VHfqryi6ORirBD2Luw+GPYplpMbCVl8L4kJ8Tbr0z6uX2NCq1VkwsI9XErOKP4brJsBB3417gto0PuG3XvPXGbp/lhGd6+HfwWPPN5A2IIUeiEcQasRvOr2Ai05yjcer+PHFaD0+7R7urky887WJKRk8H+/7Cveiw8tg1VToOVI6PxUnk+ZuTwK/wrujOpWL8/9wjak0AvhIL5NasPjmc/TRJ3iW4//UYVEoPT7tDev4cfzfRuzZM85ft19tvAXWLIhYh4sfASCW8Ct7/x952tuO05e4q+oOMb0qE8lL/dSzSwKJoVeCAdRw9+blZZ2PJr5Ag3UWb7zmEIgCTbp0/5Yj/qE1fbn1UX7iLmclveTtDamUH7UHX55Eqo2hLu+veGC8VVvLz9EgI8HD3auW+p5RcGk0AvhIK4uHrLW0pqHMydQS8WxwHMyr/bwL/Vjubm6MHNkGzKyLExYuIcbpmGf3QVfDYF5wyEzBUbMhUdWgn/tPN9v87F41h+5wBO9Gjh+B04HJIVeCAeRe/GQzZbmvOD5GjXdExm47WFIOFXqx6tX1YdJNzdh7aE45m3Jef9LJ2Hho/BJT2NB70FvwVNbofnteQ7XAGitmbnsENUqeXJfpzqlnlMUTj5ahXAgN/RpP9MZvrkd5twMD/zy90IepeW+jnVYHhnL7CXbuDXmffz2zjHmx3cbB92eBy+/Qt9j3eELbD1xkclDmuN13cpRomxIoRfCkdVsBw/+Bl8NNYp92/uhVkeoGQ7ela1+e5fsdN6vux5Oz8B3VwqWNvfg0vsV8CvanH2tNTOWHyLE35uR7WtZnUeUjBR6IRxd9dbw0BJjib51M421WMFYeLtWB6Pw1+porF5V1OUILRbY+wOsmkyly6eJCerOiNO3MNS/P08UscgDrDxwnt2nE3hzeEs83eRs3iwlLvRKKU/gMeAOIFprfXcez/EAPgSaAl7ABK31ipIeUwiRj2rN4NFVRvuBszvh9BY4vRUO/AYRXxvP8a4MNTv8U/xD2t7QngCAo3/B8leNMfjqrWHIbKrV60n9eTuZuTyKXqGBNK1eqdBIFotm5vJD1AmowLC2NUv5GxbFYc0ZfRZwEHgDeDCf54wHErTWXZRSIcBqpVQLrXW6FccVQuTH0xfq9TC+wDgzjz+SU/hziv/hpcY+5QrBLXPO+DtAxWDjN4KjObNnhn1mdJ50cUEBU4a2YNuJS4z9fhe/PN210DP0pftjiDyXyMyRrW/oqinKVokLvdY6G1imlOpVwNMGk/MhoLWOVkptAroBK0t6XCFEMbi4QGBj46vt/ca2lItwZvs/xT/ia9j6sbHPyx8GvA7tHzF67OQS4OvJ1GEteeSr7byz4jATBjbJ97DZOWfzDQJ9rO7BI6xXaKFXSvUBXstj111a68JWPAgAcj/nHHBDpySl1BhgDEDt2nnPwxVClJIKVaBxf+MLIDsLYvfBhcPQqG+BF3H7NqvGyPCafLTmKDc1Dcq3d/3iPWc5fP4Ks+8Jw9WliNcFhM0U+vuU1nqV1rpXHl9FWdYmlmsLe3DOtuuP8YnWOlxrHR4YGFj09EII67m6QY020GpEkWbqvDq4GdX9vBm3YDfJ6Vk37M/KtjBrxWGaBFfk5hbVbZFYFJOtB85+AR4BUEpVAzoBG2x8TCGEDVX0cmfGyNacupjCG38cuGH/TxHRHL+QzLh+jXGRs3m7UOqFXinVRik1P+fhu0CIUmoL8BvwlFyIFcLxdaofwOiu9fhm8ynWHIr7e3tGloV3Vx6mVU0/+jWrZmJCkZvVhV5rvVprfVeux7uuPtZaZ2it79dad9Rad5CplUI4jxcHhNIwyJcJP+7mcoqxiPmC7ac5cymVcf0ao4o6Z1/YnMx5EkKUiJe7K2+PbEP8lQxe+3UfaZnZzF51hHZ1KtOzsVxrsydyZ6wQosRa1vTjmT6NeHvFIS6nZhKTmMbMO1vL2bydkTN6IYRVnuzdgNY1/VgdFUfn+gF0aVDV7EjiOlLohRBWcXd1YcbINrSrU5mXb25qdhyRBxm6EUIUaFFENNOWRnE2IZUa/t6MHxB6batkoGGQLwuf6GJSQlEYKfRCiHwtiohm0k97Sc00OmJGJ6Qy6ae9ADcU+7xeW9gHhCgbMnQjhMjXtKVRfxf5q1Izs5m2NKrA1139gIhOSEXzzwfEoohoG6YV+ZFCL4TI19mE1GJtv6qkHxDCNqTQCyHyVcPfu1jbryrpB4SwDSn0Qoh8jR8Qivd167x6u7syfkBoga8r6QeEsA0p9EKIfA0NC+GNYS0J8fdGASH+3rwxrGWhF1VL+gEhbENm3QghCjQ0LKTYs2WuPl9m3dgHKfRCCJsoyQeEsA0ZuhFCCCcnhV4IIZycFHohhHByUuiFEMLJSaEXQggnp7TWZme4hlIqDjhZwpdXBS6UYhxbc6S8jpQVHCuvI2UFx8rrSFnBurx1tNZ5Lu1ld4XeGkqp7VrrcLNzFJUj5XWkrOBYeR0pKzhWXkfKCrbLK0M3Qgjh5KTQCyGEk3O2Qv+J2QGKyZHyOlJWcKy8jpQVHCuvI2UFG+V1qjF6IYQQN3K2M3ohhBDXkUIvhBBOzmkKvVJqpFJqq1Jqh1Jqhtl5CqKUukMptUApdcrsLEWR8/92k1JqXU7uCmZnyo9SaoJSaqNSaqdS6gullIfZmQqjlHpVKbXa7ByFUUp9qZTarJRanfN1m9mZCqKUqqWU+lUp9ZdSarlSqq3ZmfKilOqZ6//paqXUUaXUrNI8hlO0KVZK1QEmAx2ARGC+Umq41nqhucnyFQc8CewzO0hhlFJVgAlAd611qlJqGvAI8K65yW6klKoK+AFdtdZaKTUfGAL8YG6y/CmlwoF6ZucoolpAb621o6wH+AnwktZ6T87fY9fCXmAGrfUaoBeAUsoFWANMK81jOMsZ/UBgodb6sjauLn8MDDU5U7601mu01g5xt57W+iLQLdc/bjfALv+ha60vaK1fySnyvhhF324/TJVS3sAsYKLZWYrIH/hQKbVWKTXbzn+zCwa8gFFKqbXA/4Bkc1MVyYPACq11dGm+qbMU+gAgJtfjc0CQSVmcjtY6TSnlpZR6B/AGvjA7U0GUUvOA48BK4KDJcQoyDZiltT5vdpAi2g68qrXugfFb6asm5ylIbSAM+DIn71ngNXMjFUwp5QY8C7xT2u/tLIU+lmsLe3DONlEKlFI1gZ+BP7XWj2uts83OVBCt9b1AHaAzxhmS3VFKDQAqa61/NDtLUWmtx2itT+c8/AFjqNReJQCRWuuInMc/Au1NzFMUdwAbtdYJpf3GzlLofwduV0pVzHk8CvjFxDxOQynlBcwFxmit/zA5ToGUUm2UUg8CaK1TgEMYww32aDAQqJRapJRaBLRQSn1ldqj8KKW8lVKTc13cHgTsNDNTIY4AXkqpJjmP+wIRBTzfHjwGfGmLN3aaG6aUUvcCLwIZwDqt9YsmRyqUUipGax1sdo6CKKUGY1zzOJxr8yqt9X9NipSvXGPe7TCuI5wBRucUfbumlFqtte5ldo6CKKWeAx4GLgPRwGNa6yRzU+VPKdUKYxjEDTgPjNJaXzY3Vd6UUkHAbqCGtkFRdppCL4QQIm/OMnQjhBAiH1LohRDCyUmhF0IIJyeFXgghnJwUeiGEcHJS6IUQwslJoRdCCCf3/4NqmoZSgR4/AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.ensemble import RandomForestRegressor as RFR\n", "\n", "# 学習を行う\n", "rfr = RFR()\n", "rfr.fit(X,y)\n", "\n", "# 予測\n", "num_test = 20\n", "x_test = np.linspace(0, x_max, num_test)\n", "y_test = rfr.predict(np.vander(x_test, deg+1))\n", " \n", "#プロット\n", "plt.scatter(x, y)\n", "plt.plot(x_test, y_test)\n", "\n", "# sin(x)のプロット\n", "plt.plot(x_test, np.sin(x_test))\n", "\n", "plt.show()\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "どちらの方法もハイパーパラメータがたくさんあり、利用する際にそれらを根拠をもって指定できる場合は、それでよいが、多くの場合、交差検定を行って、過学習にならないようにパラメータをセットする必要がある。そのことは忘れないでほしい。\n", "\n", "sckit-learnでは、パラメータをしらみつぶしにさがすGridSearchCVなどが用意されている。\n", "\n", "とはいえ、グリッドサーチにしてもその範囲は使う側がセットする必要がある。(それが多すぎるとランダムフォレストの場合などは猛烈に長い時間を要することになる。)\n", "\n", "### 参考:決定木の可視化\n", "\n", "ランダムフォレストは、データから重複を許してランダムに抽出した複数のデータセットを用意して、それらを決定木により分類学習を行い、アンサンブル学習(多数決)により予測器をつくるものである。\n", "\n", "その一つの決定木を図示するモジュールが作られている。\n", "\n", "分類木を図示するツールとしてdtreevizが高機能なようだ。(ただし、作成には相当な時間を要する。また、Anacondaのパッケージには入ってないので、インストールツールpipを使ってインストールする必要がある。)\n", "\n", "ここではの例は説明変数が一つという単純なであるが、分類がどのように行われるかを知る参考になるだろう。" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "G\n", "\n", "\n", "\n", "node7\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "leaf8\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node7->leaf8\n", "\n", "\n", "\n", "\n", "\n", "leaf9\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node7->leaf9\n", "\n", "\n", "\n", "\n", "\n", "leaf10\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "node6\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node6->node7\n", "\n", "\n", "\n", "\n", "\n", "node6->leaf10\n", "\n", "\n", "\n", "\n", "\n", "node4\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node4->node6\n", "\n", "\n", "\n", "\n", "\n", "leaf5\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node4->leaf5\n", "\n", "\n", "\n", "\n", "\n", "node2\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node2->node4\n", "\n", "\n", "\n", "\n", "\n", "node11\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "leaf3\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node2->leaf3\n", "\n", "\n", "\n", "\n", "\n", "node13\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "leaf14\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node13->leaf14\n", "\n", "\n", "\n", "\n", "\n", "leaf15\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node13->leaf15\n", "\n", "\n", "\n", "\n", "\n", "node11->node13\n", "\n", "\n", "\n", "\n", "\n", "leaf12\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node11->leaf12\n", "\n", "\n", "\n", "\n", "\n", "node1\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node1->node2\n", "\n", "\n", "\n", "\n", "\n", "node1->node11\n", "\n", "\n", "\n", "\n", "\n", "node16\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "node19\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "leaf20\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node19->leaf20\n", "\n", "\n", "\n", "\n", "\n", "leaf21\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node19->leaf21\n", "\n", "\n", "\n", "\n", "\n", "node17\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node17->node19\n", "\n", "\n", "\n", "\n", "\n", "leaf18\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node17->leaf18\n", "\n", "\n", "\n", "\n", "\n", "leaf22\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "node16->node17\n", "\n", "\n", "\n", "\n", "\n", "node16->leaf22\n", "\n", "\n", "\n", "\n", "\n", "node0\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", "node0->node1\n", "\n", "\n", "<\n", "\n", "\n", "\n", "node0->node16\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "" ], "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sklearn import tree\n", "from dtreeviz.trees import *\n", "import graphviz\n", "estimators = rfr.estimators_\n", "viz = dtreeviz(\n", " estimators[0],\n", " X, \n", " y\n", ") \n", "\n", "viz" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 参考:Random Forest Regressor を用いたBostonデータの考察\n", "\n", "Web上の利用例紹介ページ(https://www.blopig.com/blog/2017/07/using-random-forests-in-python-with-scikit-learn/ Oxford Protain Informatics Groupによる)をもとに、サンプルとしてよく用いられるBostonの住宅価格データをRandom Forest によって調べてみる。\n", "\n", "- 評価のために、トレーニングデータとテストデータへの分割\n", "- データの標準化(Standardization): 説明変数の平均値と標準偏差がそろうようにスケールすること\n", "- 標準化したデータに対するランダムフォレスト回帰分析\n", "\n", "上記ページでは標準化を行う前と後で、主成分分析により説明変数の重要度の変化を見ているが、ここでは省略\n", "\n", "データはpandasのDataFrameとして作成している。\n", "\n", "同様なプログラム例:https://hinomaruc.hatenablog.com/entry/2019/11/14/200857\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Out-of-bag R-2 score estimate: 0.841\n", "Test data R-2 score: 0.885\n", "Test data Spearman correlation: 0.904\n", "Test data Pearson correlation: 0.941\n" ] } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "from sklearn import datasets\n", "\n", "# データの読み込み\n", "boston = datasets.load_boston()\n", "features = pd.DataFrame(boston.data, columns=boston.feature_names)\n", "targets = boston.target\n", "\n", "# 訓練データとテストデータへの分割\n", "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(features, targets, train_size=0.8, random_state=42)\n", "\n", "# データの正規化\n", "from sklearn.preprocessing import StandardScaler\n", "scaler = StandardScaler().fit(X_train)\n", "X_train_scaled = pd.DataFrame(scaler.transform(X_train), index=X_train.index.values, columns=X_train.columns.values)\n", "X_test_scaled = pd.DataFrame(scaler.transform(X_test), index=X_test.index.values, columns=X_test.columns.values)\n", "\n", "# ランダムフォレスト回帰\n", "from sklearn.ensemble import RandomForestRegressor\n", "rf = RandomForestRegressor(n_estimators=500, oob_score=True, random_state=0)\n", "rf.fit(X_train, y_train)\n", "\n", "# 予測と結果の表示\n", "from sklearn.metrics import r2_score\n", "from scipy.stats import spearmanr, pearsonr\n", "predicted_train = rf.predict(X_train)\n", "predicted_test = rf.predict(X_test)\n", "test_score = r2_score(y_test, predicted_test)\n", "spearman = spearmanr(y_test, predicted_test)\n", "pearson = pearsonr(y_test, predicted_test)\n", "print(f'Out-of-bag R-2 score estimate: {rf.oob_score_:>5.3}')\n", "print(f'Test data R-2 score: {test_score:>5.3}')\n", "print(f'Test data Spearman correlation: {spearman[0]:.3}')\n", "print(f'Test data Pearson correlation: {pearson[0]:.3}')\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAGpCAYAAADhiRM+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de7xcdXno/89DCGEjalDjJfECrW1UQEiN/rTxKFJt2kox3q2KaG3RKrQijQargvYoselpvZxWBau26qkoYhSVprUBTgXRhoYUscb+rNeNaNTE2rKBmDznj7UmTiZz3Xuuaz7v12u/sve6zHpmsvc88/1+n+93RWYiSVJVHDbqACRJ6icTmySpUkxskqRKMbFJkirFxCZJqhQTmySpUg4fdQCd3Ote98pjjz121GFIksbIDTfc8IPMXNZs39gntmOPPZZt27aNOgxJ0hiJiG+22mdXpCSpUkxskqRKMbFJkirFxCZJqhQTmySpUkxskqRKMbFJkirFxCZJqhQTmySpUkxskqRKMbFJkirFxCZJqpSBLoIcEX8NrARuLzf9GfBN4O3AEmAX8ILM3D3IOCRpUmzePsumLTu5Zc8cy5fOsH7tStatWjHqsHqyefssb7jiZnbftheApTOLufD044f2PAa9uv8DgCdk5hxARATwZeA5mbkjIl4GvBE4Z8BxSNLY27x9lvMvv4m5vfsAmN0zx/mX3wQwMclt8/ZZ1l+2g7378sC2PXN7Wf/RHcBwnseguyKXAu+MiP8bEf8beDiwOzN3lPvfAzx5wDFI0kTYtGXngaRWM7d3H5u27BxRRL3btGXnQUmtZu/+HNrzGHRi2wa8LjMfR9Ht+Fbg1trOzLyTJq3GiDgrIrZFxLZdu3YNOERJGg+37Jnrafs4ahfrsJ7HQBNbZp6Vmd8uf/xo+e+9a/sjYglwZ5PzLs7M1Zm5etmypjdIlaTKWb50pqft46hdrMN6HgNLbBExExF/HBFHlJt+naIFd3REnFBuOwO4clAxSNIkWb92JTOLFx20bWbxItavXTmiiHq3fu1KFi+KQ7YvPiyG9jwGVjySmXMR8QPgixHxY2AWeAnwIeCSiNgP/BA4c1AxSNIkqRVWTHJVZC3WUVZFRuahg3zjZPXq1blt27ZRhyFJGiMRcUNmrm62zwnakqRKMbFJkirFxCZJqpRBrzwiSdJQlwozsUmSBmrYS4XZFSlJGqhhLxVmYpMkDdSwlwozsUmSBmrYS4WZ2CRJAzXspcIsHpEkDdSwlwozsUmSBm7dqhVDWyvSrkhJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRVyuGjDkCSVD2bt8+yactObtkzx/KlM6xfu5J1q1YM5domNklSX23ePsv5l9/E3N59AMzumeP8y28CGEpysytSktRXm7bsPJDUaub27mPTlp1Dub6JTZLUV7fsmetpe7+Z2CRJfbV86UxP2/vNxCZJ6qv1a1cys3jRQdtmFi9i/dqVQ7m+xSOSpL6qFYhYFSlJqox1q1YMLZE1sitSklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlWJikyRViolNklQpJjZJUqWY2CRJlTLwxBYRr4uIq8vvT4qIayLi+oi4IiKOGfT1JUnTZaCJLSJWA8eV3wfwYeD3M/PRwJXAGwd5fUnS9BlYYouIGeCtwIZy0y8CuzNzR/nze4AnD+r6kqTpNMgW2ybgrZn5/fLnewK31nZm5p3A4c1OjIizImJbRGzbtWvXAEOUJFXNQBJbRKwFjsnMy+o2fw+4d90xS4A7m52fmRdn5urMXL1s2bJBhChJqqimLaY+OA1YFhGby59PAC4Ajo6IEzLzS8AZFONskiT1zUASW2aeU/9zRFydmS+IiJOBSyJiP/BD4MxBXF+SNL0G1WI7SGaeUv57I/CYYVxTkjSdnKAtSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjl81AFIUq82b5/lDVfczO7b9gKwdGYxF55+POtWrRhxZBoHJjZJE2Xz9lnWX7aDvfvywLY9c3tZ/9EdACY32RUpabJs2rLzoKRWs3d/smnLzhFEpHFjYpM0UW7ZMzevfZoeJjZJE2X50pl57dP0MLFJmijr165k8aI4ZPviw4L1a1eOICKNG4tHJE2UWnFIs6pIgDUbt3LLnjmWL51h/dqVFpNMIRObpImzbtWKQxLW5u2znH/5Tczt3QfA7J45zr/8pgPHa3rYFSmpEjZt2XkgqdXM7d1npeQUMrFJqoRWFZFWSk4fE5ukSmhVEWml5PQxsUmqhPVrVzKzeNFB22YWL7JScgpZPCKpEmoFIpu27LQqcsqZ2CRVRrNqSU0fuyIlSZViYpMkVUrbxBYRR0TE0hb7njKYkCRJmr9OY2wPBdZExBHAveu2fwN4DvCJAcUlSdK8dOqKTCCApwJXAV8HTgGOHmxYkiTNT8sWW0Q8CXgtsAS4CzALHAXcDvzXUKKTJKlHLRNbZv5DRHwfeCLwW8DTgPsAy4EHDyc8SZJ60808ttuBsylabQl8HPgxsHqAcUmSNC/tuiIPA34EfCUzr4+I1wAn1x2SEfGRzHzWoIOUJKlb7Vps/x9wIfCViLg/8B2KcTZJamrz9lmXtNLItRtj+zywNiJ+HngucC7wUeAz9YcNNjxJk8IbfWpcdFx5JDO/lpl/DPwc8AXgk5n5ifLrkwOPUNJE8EafGhdti0ci4mENm64HHhoRtZ9vycw9gwhM0mTxRp8aF52qIp9J6+7GoJgCcFJfI5I0kZYvnWG2SRLzRp8atrZdkZn5BuCuTb62lft+MPAIJU0Eb/SpcdHNPLZ3NNn244i4H8XKJJLkjT41NjqNsd0CfBb4JYqFj+9GMWF7G/AIYP2A45M0JP0o1fdGnxoHnVpsO4GXAm+iKPX/BYrux4cA52TmVwcbnqRhsFRfVdKp3P924C8oSv1fDpwGPB/YDbwpIt4cdSWSkiaTpfqqkk4ttlcAJ2bmZRFxUmbuiIhXUqxA8hzgkZnpJG1pwlmqryrp1GJ7FbA1Ip4OvCEiFgGXUYyvXQesGXB8koagVUm+pfqaRJ3K/V+cmT8C/gN4Vmbuy8xvZeabgccC/ziMICUNlqX6qpKO5f4RcQRwc2beWf58WmZ+iuKmo98ZcHyS+qRd1aOl+qqSbuax3QJcFxG/CzwAeCXwKeBJFEUlfzK48CT1QzdVj5bqqyo6LoIM3JSZpwN/SzGfrWYdcMVAopLUV1Y9app0k9hqVY8Hyvoj4gRgcWb+20CiktRXVj1qmrRNbGU1JBFxOnAvigWPlwFvBH5v4NFJ6gurHjVNOrXY7lX+uwxYTLEA8uLy68EDjEtSH1n1qGnSqdz/3eW/fwV8F7iWopjkGcCFEXFkq3Mj4lURcV1E/EtEvDcijoiIkyLimoi4PiKuiIhj+vhcJLWwbtUKLnraiaxYOkMAK5bOcNHTTrRYRJXUTVXkITLzjoj4K+C3gPc17o+IewF3B9ZkZkbEh4GnUHRhPqdcweRl5c/nzDt6qeL6sTBxjVWPmhbdFI88PCI+SZHEvsjPiki2Ao9vdkJm/iAz/6hMakdTJLmvArszc0d52HuAJy8oeqnCaiX6s3vmSH5Wor95++yoQ5PGWjeJbQXwzMy8NTO3A5sAMnM38AftToyIDwFfp1ih5Gjg1tq+csJ30xZjRJwVEdsiYtuuXbu6eiJS1ViiL81Px8SWmXeUXY+LIuJemfmZun0/7nDu84AHAY+huNXNvWv7ImIJcGeL8y7OzNWZuXrZsmVdPhWpWizRl+anmxYbEXE2sAO4LCK+GhEXtLtdTUScHBFnAmTmbRTdkHcFji7nwAGcAVy5oOilCrNEX5qfjoktIn4VeCjw8Mw8BTieouX1qjan7QR+uexO/CfgWOBi4IXAJRFxLXA68PqFBC9VmSX60vx0UxX5IuBFmbkfIDP3RsS5wOcjYlNte73MnANe0uSxbqTolpQqoZ9Vi43aLUw8yOtKk66bxHYf4OUR8WXgC5n5o8y8MyKuomiJ/ccgA5TGVTcLCy9UsxL9YVxXmmTdjLHtBv4dOBm4NCI+ERG/kpl/mJkmNU2tUVUtWi0ptddNYvtn4L8z86LMfBLw+8DvRETbUn+p6kZVtWi1pNReN4nt7cCrIuKRAJn5TYq5bGsHGZg07kZVtTjJ1ZKbt8+yZuNWjtvwadZs3Opkcw1EN/PYbgOeDfxuRHwuIq4GXg2cNeDYpLE2qqrFSa2WdCUVDUtXa0Vm5h5MZNJB2lUtVvG6C9VubHDcY9dkmdciyJIKo1pYeFjX7ee0AscGNSzdTNBeERFHDCMYSeOj312Hkzw2qMnSTfHI3wCfiIg/i4j7DDogSeOh39MKJnVsUJOnm8R2RGb+OvAJ4IMR0WxFEUkV0++uQ292qmFpO8YWEfcGEiAzr4mIpwLPjIjlmXnBMAKUNBrLl84w2ySJLaTr0Judahg6tdj+HHhERDw8Is4C5jLzicBRttw0LaZ17pVdh5pUbVtsmfm8iHgY8DbgHpn5iHLXq4HPRsTHM/P7gw5SGpVpXpdxUqcVSJGZnQ+KWAS8C/hSZr6t3PZY4JZBrxe5evXq3LZt2yAvIbW0ZuPWpt1xK5bOcO2GU0cQkSSAiLghM1c329fVjUYzcx/FBO39dds+5yLIqjrnXkmTp6vEBpCFdwwyGGncOPdKmjxdJzaAiHClEk0VCyikydNTYgO+OJAopDHl3Ctp8vTaAouBRCGNMedeSZOl18TWuYRSUt/0cxFiaVo4ZiaNqWmeQyctRK9jbJKGpN+LEEvTwsQmjSnn0EnzY2KTxpRz6KT56TWxWRUpDYlz6KT56bV45DcGEoWkQ7gIsTQ/LRNbRCwGNgGvysw7ATLzu8MKTJJz6KT5aNcVeQpwE8Uta4iIq4YRkCRJC9EysWXmPwD7gHeXm44cSkSSJC1ApxuNvj8irgN+eUjxSJXj6iHScPVSPLIkIt4IfAnYXBt3k6ZJLUnN7pljUQT7MlnRJln1snqICVDqj17K/X8K7ACOB66LiBMHE5I0nmpJqnZH7X3l3edryWrz9tlDzul29ZD6x84OjympvV4S277M/FhmXgCsA94bEY67aSps3j7LeR/ZcUiSqmm11FW3q4e4fJbUP90ktn8v/z2wsn9mfgd4J3DCIIKSxkmtNVVrobXSLIl1u3qIy2dJ/dMxsWXmmeW30bD9vZm5bSBRSWOkWWuqmWZJrNvVQ1w+S+qfXroinzqwKKQ+27x9ljUbt3Lchk+zZuPWBY1VddNqarXUVbd34Hb5LKl/uq6KzMxbBxmI1C/9vo/Z8qUzBwpGmmlWFdlrhaPLZ0n9E9lh3OCggyMWZWbnPpk+Wr16dW7bZo+nurdm49amiWjF0hmu3XBqz4/XmCihaE01a3nN5/j5cnqApllE3JCZq5vt63UR5NdExI8y8y/6EJc0EP0uxOi1NdWuwrFficf5cVJrXY+xRcS9gdOBvxpcONLCLaQQo9XY3LpVK7h2w6n8+bNPBuDcS29sOXY3jApH58dJrfVSPPJ64JWZefuggpH6Yb6FGJ2SQLdJYhgVjgudH/eKNolZmnRtuyIj4gXlt0cBTwS+GBHH1R1yaWbeMajgpPmYbyFGqyRw3kd2cO6lN3JYuYRW4/7GLsb1a1c2HWPrZ4Vjq4KWbufHwcKLaqRx1WmM7a5137+j4WfwjtoaU/O5j1mrJFBLZq0maDc7b8nhhx1IbMcctZgLfvP4viaPbpNnp4rOfo/9SeOg0+r+FoloanRKAu3Oq2lWEXn73v19ia9et63SZgmwkaubqGp6rYqUJl6rKsFukkCjxlbSMCoia7ppldYnwFZJ29VNVDUmNk2chZSvd1MmX3vsZmNqAIsi2J/Z9NrjuOZjLQG2ml/n6iaqGhObJsprN9/Eh67/1oEVuXstgOjUoqpvBc1nonW3RR2j4OommhbzSmwRsQI4OzPP73M8Ukubt88elNRqeunq66VF1SwRPOEhy9i0ZSfnXnpj08QwjIrIhZhPUY00aTomtog4CfhyZu4tf14KfAq4YMCxSQfZtGXnIUmtptuuvl5bVO1acM1ai7aKpNHrNI8tgCuBXRFxCfCXwAeAN2bmJ4cQn3RAu+R1WATHbfh0x0TSTYuq1Rhet4Uhtoqk0epU7p8R8Q3gV4BzgK8CH8zMjw8hNukg7crxa0UencbcOrWo2rXKWiXW2T1zrNm41RaaNCY6ru4fEZ/PzMeU3z8K+BPgWZn5/SHE5+r+OqBZMUcr813Jv92dAYCm+wIO6iIdxEr+kg7WbnX/btaKPLC6SGZ+ETgPuCIilvQpPqkrzW7a2cp8y+vbFZc0W4OyMalB88WIJQ1PN1WRByW/zLwhIv4aeB3w2oFEJbXQOH7VqoU13/L6dsUlzboxW3WNupqHNDrdtNh+pcm2dwPHNdkuDdV8V/Kf7+PVbl/z9Y1P5toNp7ZsNY7DvDVpWnVMbJn5kybb9mXm8wYTktS9Zt2TCxnf6vXx+p1YJS1cx+KRAwdGXJWZTxhwPIeweGR6TcqdnyclTqlK2hWPdDNB+39l5nnAkX2PTGqhm8nQ9ceOMrE4b00aL90Ujzxm4FFIDbqdDN1LAhyEUSdVSYfqZa3IJRHxRuBLwObMvHNAMUldr+nYy21imiWh2mMM6k4Bkoavm6rImp8CO4Djgesi4sTBhCS1rips3N5tAqwlodk9cyRFElr/0R2sv2zHQdvOv/wmNm+f7SrGdklV0uj0ktj2ZebHMvMCYB3w3ohw3E0D0W21YbcJsFkS2rs/2bvv4OKpXhLTON57TVJ3XZH/Xv574B0gM78TEe8ETgAsWVTftVvTsb5LcelRizkM2F937uLD4pAE2EuyaTXputE433tNmmYdE1tmnll+Gw3b3zuQiKRSs2rDxnGt3bftPfTEOHTT3WcWs2euybFNRHmdTuNk437vNWla9dIV+dSBRSF1qVmXYqO9+/KQ7sRokuxayfI6nfR7crik/ui6KjIzbx1kIFI3uu1SbDyuacuuD9dxDps0fnppsUkj1+34Vf1x3VY5zuc6ksaPiU0jtXn7LGs2buW4DZ9mzcatHZNQs2rJRvXjXLUxuU7Htzpf0uQxsWlkms0t6zSPrH5cC2BROXhW+7dxnKvTmFzteMfJpOroZeWRnkXEs4BzKSZ3fxd4IfALwNuBJcAu4AWZuXuQcWg0Oi039YYrbu561ZB6tX31FYn7Mg9qaa3ZuJVbyoTZyuJFcSAmE5lUHQNLbBFxD+BVwP/IzLmI2AT8LvBS4DmZuSMiXga8EThnUHFoNDotN7V5+2zLgo5uCjdarfrxhitu5va9+ztWTgKH3vpaUiUMrCsyM38EPDYza+9ShwO3A7szc0e57T3AkwcVg0an03JT7crpGws/mo3BtUp+u2/b211So1h5xOWvpOoZaFdkZt5eLrv1Foquxy8Bt9btvzMiDokhIs4CzgJ44AMfOMgQNSCdlptq1yqrdSe+dvNNfPD6bx3YXlvfEVqv+tGvOCVNroEWj0TE/YGPA3+XmS+lSGr3rtu/BDjkLgGZeXFmrs7M1cuWLRtkiOpCr5WL0HkNx1b7l84sPtBVWZ/UavbuTy785M1NqyPbzcFuNUH77jOL25wlaRINLLGVLbX3A2dl5pUAmfk14OiIOKE87AzgykHFoIWbT+UidF7EuNX+C08/HmjfVblnbu8h1ZFB6yGzmcWLmDm8+a96LyuSSJoMg2yxPRF4KPCBiLi6/Ho9RWXkJRFxLXA68PoBxqAFmu+tWTotN9VpfzddhOtWreDaDaeyYulMy6RWe9y5vfub7t/T44okksbfwMbYMvNTQKsaau/KPUK93PV5Ibdm6VRG325/uzG0Y446uPuwVSwBXLvhVKBI0K7EL00HJ2hPmV67Fru931m/rV+7ksWHNe8nzDx4max2MdbGB2f3zB0yBucKI1I1mdimTK9di93e8LPf1q1awbMf9YCmBSF75vYelIxbxfiEhyw7kMShGIOrPZ4rjEjVZWKbMr12LY7q1iybt8/ysRtmW46d1SfjVjFe9ZVdhyTxLPdfu+FUk5pUUQOdx6bxM5+7Po9iyalu7rtWn4wb77jdakyt8TxJ1WOLbcp06lqcz5y1Qegm+TSuUNI4dtiqkt+CEanabLFNmcaWTX1VZKf1HYep08oijeN8zVp4zboxawsfS6ouE9sUatW12K6wZNiJbf3albzi0htb7m8c5+u6e9GFj6XKsytSByxkzlq/rVu1gqUtlrtasXTmkETbbfeiCx9L1WdimzCDHAMb1Zy1Vk476X5dzz3r5s7aNRaPSNVmYpsg8123sVujmrPWTLNy/wCe/ojm3ajNSv5btfgsHpGqzTG2CTLoMbB2hSXD1qoY5Kqv7Gp5TuPYYWMxDLjaiDQNTGwTZBhjYKOYs9ZMP57rOCVqScNjYpsg85lcPan69VzHJVFLGh7H2CZIq5trPuEhB9+MdVwmWc9H/aLFzTQ+V0lqZIttgqxbtYJt3/wRH7r+WweKKhL42A2zrH7QPcZuknW9bm6V02xMrFG7MTZJAltsE+eqr+w6ZI5x/YLA870x6CB1W83Z6/qQktSMLbYJ06moYpwmWdd0W83Z6/qQnfRyQ1VJ1WGLbcJ0mkQ9bpOsoftk2ynGXkr1Bz3nT9L4MrFNmE6TqMdpknVNt8m2VXEM9H4fuHHskpU0HHZFTphOc7MWMndrUF1369eu7GqidD/nnY1jl6yk4YjM8V7ufPXq1blt27ZRh1F5rVbp6Nfdsoc93tVqykDt7tmSJltE3JCZq5vts8UmYDjLdQ2zcKPbVqKk6jGxCVhY1904Vh+6nJY0vUxsAua/hNW4TgivXX/UMUgaPqsiBcy/mtLqQ0njxhabgPl33Vl9KGncmNh0wHy67qbpjgOSJoNdkVqQcZwQLmm6mdi0YEcu/tmv0dKZxX2b+yZJ82FXZMUMs/S+2aTuO366fyDXkqRumdgqZNil94Oe1K3xM45zFqVGJrYKaZVozvvIDqD/yc2KyOkyznMWpXqOsVVIq4SyL3Mgt2wZx1vkaHCcs6hJYWKrkHYJZRBvQFZEThdb6JoUJrYKaZZo6vX7DWjdqhVc9LQTWbF0hqD3e6ZpsthC16RwjK1CagnlvI/sYF+T2xEN4g3I9Rinh3dM0KQwsVVMLcm0ewOysk3z4R0TNCm80WhFtUpezeaeQTGx+sLTj/dNStJE8EajU6hVF2GzyjaAPXN7D5Ru147zU7mkSWRimzLtCkjm9u7jDVfczO179ztXSdLEsipyynQqINl9217nKkmaaCa2KbN+7UpiHud1M1Vg8/ZZ1mzcynEbPs2ajVv7PiFckrphYpsy61atoF250NKZxU23d2rp1YpSZvfMkfysC9PkJmnYTGxTaEWLJHXMUUVl5HxWE3G5JUnjwsQ2hVothXXBbx4/79VEXG5J0riwKrLi2k3Grm2/+8xiIuDcS29k05adrF+7kms3nNrTdZYvnWG2SRJzuSVJw2aLrcLajXutW7WCazecyp8/+2Tu+Ol+dt+2d0FjYy6ILGlcmNgqrJtxr36NjbkgsqRxYVdkhbUa35rdM8eajVu5pWzJtTq31zUlXRBZ0jiwxVZhrca3Ag50T7Zy95nFlu9LmkgmtgprNu4V0DahQTE2FoHl+5ImkomtwpqNe7VLavVjY3tu29v0GMv3JY07x9gqrnHca83GrU3L8lcsnTmoxH/Tlp2W70uaSLbYpky3ZfmW70uaVLbYpky3d0H2bsmSJpV30JYkTZx2d9C2K1KSVCkmNklSpZjYJEmVYmKTJFWKiU2SVCkmNklSpTiPbUDarYzf66r5kqTumdgGoHaDz9oiwrWV8Wta7TO5SdLCmdgGoNPNO1vtM7FJ0sKZ2Oh/12CrFfDbrYzvqvmS1B9TXzxS6zbs5w01W62Av3zpTNt9kqSFm/rE1qrb8LyP7OC4DZ9mzcatPSe5divju2q+JA3WwLoiI+IZwLOAR2fmA8ttJwFvB5YAu4AXZObuQcXQjVZdgPvKxaHnU9zRzcr4VkVK0mAMbHX/iHg8cDPwpcy8b0QE8GXgOZm5IyJeBjw0M89p9ziDXt2/1Y03GzXeiFOSNDojWd0/M6/JzB/UbfpFYHdm7ih/fg/w5EFdv1vNugabsbhDkibDMKsi7wncWvshM++MiKbXj4izgLMAHvjABw40qMZuw8MiDnRD1qsVdzi5WpLG28BvNBoRt5ZdkT8P/HVmPrbcvgS4OTMf3O78Yd9otHFyNRTFHRc97USAlvsGmdxMppJ0sLG40Whmfg04OiJOKDedAVw5rOt3a92qFVz0tBNZsXSGoBhbqyWuThOvB2EQ0xEkqcqGPUH7hcAlEbEf+CFw5pCv35V1q1Y0bRHNZ+L1QrVLprbaJOlQA09smXnfuu9vBB4z6GsOyvKlM00rKAc5uXoUyVSSJtnUT9DuxSgmV7tSiST1xsTWg3bjb4PiSiWS1BsXQe5Rq/G3QV4PXKlEkrplYpsAw06mkjTJ7IqUJFWKiU2SVCkmNklSpZjYJEmVYmKTJFWKiU2SVCkmNklSpZjYJEmVYmKTJFWKiU2SVCkmNklSpZjYJEmV4iLIbWzePjs2q+qPUyySNM5MbC1s3j7L+ZffxNzefQDM7pnj/MtvAhh6QhmnWCRp3NkV2cKmLTsPJJKaub372LRl53ElSSwAABAKSURBVFTHIknjzsTWwi175nraPkjjFIskjTsTWwvLl870tH2QxikWSRp3JrYW1q9dycziRQdtm1m8iPVrV051LJI07iweaaFWlDEOlYjjFIskjbvIzFHH0Nbq1atz27Ztow5DkjRGIuKGzFzdbJ9dkZKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaqUyk/Q9nYvkjRdKp3YvN2LJE2fSndFersXSZo+lU5s3u5FkqZPpRObt3uRpOlT6cTm7V4kafpUunjE271I0vSpdGKDIrmZyCRpelS6K1KSNH1MbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaoUE5skqVJMbJKkSjGxSZIqxcQmSaqUyMxRx9BWROwCvjnqOHp0L+AHow6iR5MYMxj3ME1izGDcwzTMmB+Umcua7Rj7xDaJImJbZq4edRy9mMSYwbiHaRJjBuMepnGJ2a5ISVKlmNgkSZViYhuMi0cdwDxMYsxg3MM0iTGDcQ/TWMTsGJskqVJssUmSKsXEJkmqFBPbAkTEMyLiIxHxrbptJ0XENRFxfURcERHHjDLGZlrE/aKI+EpEXF1+vX6UMTYTEc+KiM9HxD+V8R81Ia93s7jH+vWOiFdFxHUR8S8R8d6IOGJCXutmcY/1a10vIl4XEVeX34/96w2HxDwer3Vm+jXPL+DxFBMSby1/DuDfgJPKn18GvGPUcXaKu9z2BuBXRx1bm5jvAWwDZsqfNwF/MO6vd4u4f3+cX+/yd+NN/GwM/sPAMyfgtW4V99i+1g3xrwbeC1w9Qe8lB2Iufx6L19oW2wJk5jWZWT/L/heB3Zm5o/z5PcCThx9Ze03iBjgWeE75KevjEXHcCEJrKTN/BDw2M+fKTYcDtzPmr3eLuOcY49c7M3+QmX+UmRkRRwN3B77K+L/WzeL+EmP8WtdExAzwVmBDuWns30uaxAxj8lqb2PrrnsCttR8y806KN7JJ8GXgbzLzFOBtwIdGG86hMvP2iDgyIt4GzFC8aY39690k7vcyAa93RHwI+Drwj8DRTMBrDYfE/RUm4LWmaMm/NTO/X/48Ce8ljTHDmLzW4/ZCTbrvAfeu/RARS4A7RxdO9zLzLXXfXx0Rx0ZEZNm/MA4i4v7AJcDbM/PKiPh5JuD1boy73Dz2r3dmPi8ijgI+AOxmAl5rOCTuM8f9dzsi1gLHZOZldZvH+r2kRcxj8z5ii62PMvNrwNERcUK56QzgyjanjI2IeHVEPKD8fjXw7XH5wweIiCOB9wNn1ZLDJLzezeIut4/t6x0RJ0fEmQCZeRtFN+RdGf/XulncS8f5tS6dBiyLiM0RsRk4AbiA8X69D4k5Iv5mXF5rJ2j3QUTcmpn3Lb8/GXgnsB/4IcUnxt2jjK+VhrhPBTYCd1B8Mvy9zPzqKOOrFxGnAe8G/r1u81bgk4zx690m7s8xpq933djJIyjGA78DvJhi3GecX+tWcT+aMX2tm4mIqzPzlAl7L6nFPBbvIyY2SVKl2BUpSaoUE5skqVJMbJKkSjGxSZIqxcQ2JSLimIh4eUQ8bNSxaLQi4lcj4ohRx9FvEfHLEXF43c+/2uP5S8p/71+3bUWT4+4VEYsWEqsGy8Q2QSIiejj2LhHx6bpNzwTuC5ze5NiHlaXSQxERj4qI+8zjvJa/rxFxdkS8sM3+IyPij3q83iUR8eAOxxwZEcsjYnVErOzl8bu4/ryfb5vzXgQ8EPjTBcT1f+bz/9fwGMfVLZRb+9oREd+OiLuUx0REvC8iHhIRf1cec2vdY1xYN88L4EJgRTmf7WTgtbXvy/mETeMo/70LcF2Z8C+PiPuWifHPmpz2AYqVWIiIF87n/2FQIuJ+rZ7rNHHlkTEQEe+mmHtT7+rM/MO6Yx4EXBER52TmNV087O8Cj4yIz5U/fxt4X2b+fZNj/xb4NYq5P42x/RrwGmARxQehALL89/SG5XTqz3sc8PXM/HaT3b9NsUDt91oFHxF3o1gSKcuv/cCxEfEbmbm91XltnAzcp3zsZwLPoFg093BgQ2Z+vsk5Py2PXwLcJzO/Vb8zIl4DPI7itX0c8H8oFoFtfC5/QzEHrN7nM/PcumP6/XxbXfdlwEsoFpCez2M+DHgq8MDyc9adFPOrvl13zF0oFnp+JHAM8H3gvZm5pXZMZn4dOCUiLgCuAY6iWAvxDZn53+UxGRHrgSdQ/K68HHhXRBwLnA38V901HwzsLJ/vSuBIYAnw2PKQb1GsLVr/XM4EdlEsv3Uf4AaK34stFBOQXwG8MSKWZ+Yt5Tn3AO6SmT/u8DotAp4CPB1YX3f+I4EXAs+mWEP0K+X2pcBfAfej+Ft7SWbeWH6YfTNwavl8NmXmh8pzngX8YXn81Zl5Xvn9m4FXtouv8ha6irJfw/uiWJX/G8DD6ratBN7UcNyDgH+m+AOE4o/2shaPeTzwD22ueTjF+oaLG7b/M3BEm/MupPjDbbbvXcApPT73o4CbgCUt9p9NMRm01fm/Bzy//P444BcoVtM4CjisxTlvAz4LfBz4/TaP/Urg7D7/Xy/o+Q7o9+/w8vU4uW7bFuDIup+PAf4eeBLFG/v68rn8JXBOk8e8G/C/KVc3adj3S8Azyu8/STH5+ssUC+3+afk7dkK5//XAi8rvH0ORpG4G/gl4ZZPrLgE+3vD7eh6wjiLBvL78/i3A7zT8X78O+IvytWj8OrIu3hdQJOUrgJ8rt58EPJhiBf+H1D3uJbXXB3g4cGP5/fOAyyg+SN6tfP73o/gb30mx0HMAlwJPL895C/DwYf5ujNuXLbYRi4jzKd4AGt0f+KPM/Ovahsy8JiIemj9bKR6KP/Y31z3eMooW2E8pWinXUnx6fn+LEJ4LfKBslZyUmV+s35mZPy0fqz7mnwd+mMXCrPXb30fxxgbwEOApEfFj4Fcyc1+L69fOvQdw9yw+yTdzDnBJZt7R5mFeEREnUnzqb2wN3p9ySaI212h0O8Wb4r+2iDmANwJPBG6NiMdSLAD7mXL//SiSYs0iiuRQew53ZObjW1x73s93gddtquwWfRfF4ryPBm6MiMcD/5aZ9S2hPwFen5nXly3aHZl5W0ScA/xrRLyL4k35PQ2XWELRIwHF6/50isR0YUR8Dfhvijf4xhYoUdyn7Fzg3LK7+QvAJ4DLgeUUrZ1GjwcaW+nPBX5C8bvy3+V5y4A/L69zT4rei43Aa4Hfysy/LPe9Aziv7m9iFUWr67vAVbULZLlaf5NRhd+g+DslM/81Iv6z/Ds7Dbg4i4z1nxFxWXns4cDHsmw5lr0+LwI+RvF//yyg6e/tNDCxjVhmXgRc1Lg9Iq6j6JpqPH6u7piNwLcy85/qDnkbxafkL1MkrGsouoRe2+Qah1OMuT2K4lPfOyLi9zLzXzqE/SaKlb0b/QFFd+U6ik+1Z2fmZzs8FlHcYuSy8jEPSToR8SiK7rND3tQaXFSe/+mIuAJ4c2buLffdg+LNqhc/Kc9rFvMM8D5gH8Wb6g6KN8EPRsQPM/ML5Zvao+vOeQZFa+eQ/4uGx17Q8+32uhGxHPhMh2s8qnyzPpsi0ZwFXBTFnQpOAJ7fcPzDMvP68vtTgYsBMnNfRPyAYuHc/x84pS6OJ1K07i+MiKcCp2XZHRkRT8/MvRGxmKKlc7cmMT6N4vcHioTyBYoFhB9M3ULCDY6nuN9ZvbeW234L+BrwRQ5Oir9O0VIjM3dHcVPNd1N0G5/U8EHv+RR/f98E/iwzb24RR83hDR9Yv1vGftAq/3Xbo8V2KP72X93hepVmYhtD5WD0jsz8Tov9d6P41HhPik+ZB2Tmc+uOez7FL/na8hNfo98E/r72BxURZwAfjYg1mflfTY4nIl5KsRRbs6T7n2W//6MoulY6iqLg4n0cvPJ9/f5HUCS8DwKbI+KMbLNeXtmqfRzw4rqkBkUCWl+2kJ9bfqBoFs/RFPeXeiTwAGBlRHwvMxvfBJ9A8WZ6S/l8v56Z34qIj1J0N32hycM/nOJWOy318fl2vG4W4z4nt4un7ti31/346oj4R4oWw3cbDt1XPo8HAXNZ3vevTEwrKMa0mipbvH/AwfcdW14m5n3AD2gYJyu9n2ItxaV1255EkXiPpOj2a7SEn7Vgofi/PIEiGd6PYuzwRxRjc18EyMwPRsQpFDfXBLiRorv0DuCgD4NZjINfExHHA5dExKsbPoA2mouIJXUt9PtSjCsetMp/uf2bFIntuIbttVb7HRTDB1PLqsgxExEvoCiueEWTfSdGxAaKOzJ/haIrpGkXX9lt8mGKgf2bmux/KUW3ys9FxKUR8VngHRRvEG9pcvx9IuIvKbrdzmiy/5iI+BjwcxTdaJ0sj4g/oeiSeklmfrjh8Q6PiJdRdH89JzNfT3Efs6uiXD28lcy8LTPf0bD5J8A9M/ObwG9ExAnlNe5Wdt/WvJii2OE0irGWn1C0UD5SdvHVrvEZinGUPwb+gaKo4ZEUr80hBTpRlJCfBmxuFvMAnm9X1+1VRCyKiD8Frqt1wzX4Xpmc3wK8vTznCMp7c7X4gFUzCzy71lor1T6c/T1F9/Y24MfUJasmfwN7KP5vPgx8lGJcutm1lpfxvYCitfY75dcjKYqpaj9fHBG/3uQxrqRo0Z9O3esbRaXs2RGxuGypXUldC7qFT1F0JRIRD6UYb/wPii7VF5fbj6JonV5J0dJ+akTctTz/t8tjofgA0fRD8dQY5QCfXz/7omh5XUXxxjbTZP8xFIPQLweWdXisl1P84a9pc8wZFAPTj6f45Leo3B4UYwX1xz6SopDhTFoUWpTH3a/89xKKxPuLLY57DUUL4oWtHo+iwvDNja8F5YB+k+PPBl7YJrazKaofoWhR/S1wPcUq+2+pO24lxer7/7fcX3tOD6dJsUz5/3UVxRvu5cCJTY55MUUxw8lt4uvr8+32uj3+jv6P8rm2K9J5AMUb7yvL36X/BWwHzq/9jjU550nAhU22H0FRpLSYokX1EIoPESvK/Rfys+KRdeXv09byd7/x66kNj70CeE+LeDYAv9Zi3ykUhSv1j72LokjlauDJ5XF/SNGi217uu0/D41zNwcUjx1Ak488D19X+z+pew23la/G8unOeVz7+F4A/rdv+2xQfjvr+PjUpX67uPybK7qRvZENJ+Twf6xHAv2brrqn5POZhmbm/X4/XxfV6ujlh+Wk28+Bxivr9dwH2t9o/SOWY2ZeyuEdYq2P6+ny7vW4voiip/6/MvLXjwQef840sipAa9z2I4sPA3YH/mZnvb9i/iqKH4HPAUzJzQxTz044C/ifFmObjM/NHEbGOoivytMx8RpexvZMioX6vYfsGiqrEv2tyzinA6szsOA8wIu4NrMvMi7uJpx+iKPK5lKK7vW9//5PGxCZp7EXEouxQWTuPx1xKUbH7sX4+7ihFxC9RvK/fMOpYRsnEJkmqFItHJEmVYmKTJFWKiU2SVCkmNklSpZjYJEmVYmKTJFXK/wOhZNNsQ1DIUAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(7,7))\n", "ax1 = fig.add_subplot(111)\n", "ax1.scatter(predicted_test, y_test)\n", "ax1.set_xlabel(\"ランダムフォレストによるテストデータの予測値 (単位$1000)\")\n", "ax1.set_ylabel(\"テストデータの実測値\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "上記では、引用したページと同じ木の数(n_estimators)を500にしているが、もっと少なくてもさほど違わないだろう。\n", "\n", "以前に紹介した[単純な重回帰分析](https://8tops.yamanashi.ac.jp/~toyoki/lectures/simulationMethods/multiple_regression.html)の結果との比較をしてみてほしい。\n", "\n", "\n", "以下にgraphvizによる図示プログラム例を示す。(巨大な木で、画像のダウンロードに時間がかかるので出力は削除した。)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "from sklearn import tree\n", "from dtreeviz.trees import *\n", "import graphviz\n", "estimators = rf.estimators_\n", "viz = dtreeviz(\n", " estimators[0],\n", " X_train, \n", " y_train,\n", " target_name='price',\n", " feature_names=boston.feature_names,\n", " # class_names=[],\n", ") \n", "\n", "viz" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "多数の大量な説明変数をもつデータの分類、回帰に対する決定木ベースの手法は、最近も研究が進んでいるようだ。\n", "そのなかでもXGBoostなどが有名である。モジュールも存在するので興味ある人は試してみてほしい。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.11" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": true, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 4 }