{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 多項式曲線フィッティング (2): 過学習を防ぐ「正則化」項とscikit-learn利用の暫定まとめ\n", "\n", "
\n", "\n", "## (先週話したことの補足)\n", "\n", "数学的な詳細の理解には時間がかかるが、\n", "\n", "> **2乗誤差最小化問題という多変数関数の極値(最小値)問題が線形代数の(行列)計算に帰着できる**\n", " \n", "ということに注目してほしい。これは評価関数が2次関数の線形結合になっている場合に限られるが、「計画行列」の活用は後のカーネル法などにも引き継がれる。\n", "\n", "(注) 関数の極値問題は勾配法(一変数の場合はニュートン法と呼ばれる)による繰り返し計算を行うことが多い(たとえば、https://fussy.web.fc2.com/algo/math10_extreme.htm)。\n", "\n", "多数の局所的な極小がたくさんある場合は困難であり、様々なシミュレーション的方法が行われている。\n", "\n", "\n", "## 今週の話\n", "\n", "\"IPython Interactive Computing and Visualization Cookbook\" (O'Reilly, 2018)のサンプルプログラム8.1を例に\n", "(現在、原文はhttps://ipython-books.github.io/ にて閲覧できる。)\n", "\n", "模式図はPRMLより。\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2乗誤差最小化 (復習)\n", "\n", "説明変数$x$に対する目的変数$y$について、$M$次関数でのフィッティングを考える:\n", "$$\n", " y(x, {\\bf w}) = w_0 + w_1 x + w_2 x^2 + \\cdots + w_M x^M = \\sum_{j=0}^{M} w_j x^j\n", "$$\n", "データ$\\{(x_n, t_n)\\}$ ($n=1,\\cdots N$)があったときに、\n", "2乗誤差\n", "$$\n", "E({\\bf w}) = \\frac{1}{2} \\sum_{n=1}^N \\{y(x_n,{\\bf w}) - t_n\\}^2\n", "$$\n", "を最小にするよう${\\bf w}$を決める。\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "説明用に、PRMLに載っている図を載せておく。\n", "\n", "最小2乗法の模式図:\n", "\n", "\n", " | x^4 | \n", "x^3 | \n", "x^2 | \n", "x^1 | \n", "x^0 | \n", "y | \n", "
---|---|---|---|---|---|---|
0 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "0.000000 | \n", "1.0 | \n", "0.868819 | \n", "
1 | \n", "0.004796 | \n", "0.018224 | \n", "0.069252 | \n", "0.263158 | \n", "1.0 | \n", "1.226901 | \n", "
2 | \n", "0.076734 | \n", "0.145794 | \n", "0.277008 | \n", "0.526316 | \n", "1.0 | \n", "1.057635 | \n", "
3 | \n", "0.388464 | \n", "0.492054 | \n", "0.623269 | \n", "0.789474 | \n", "1.0 | \n", "0.085984 | \n", "
4 | \n", "1.227738 | \n", "1.166351 | \n", "1.108033 | \n", "1.052632 | \n", "1.0 | \n", "-0.646647 | \n", "