Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Last revision Both sides next revision | ||
|
optimization_set [2016/05/20 21:39] nikolaj |
optimization_set [2016/07/24 21:21] nikolaj |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| | @#55CCEE: context | @#55CCEE: $ B $ | | | @#55CCEE: context | @#55CCEE: $ B $ | | ||
| | @#55CCEE: context | @#55CCEE: $ \langle Y, \le \rangle $ ... Non-strict partially ordered set | | | @#55CCEE: context | @#55CCEE: $ \langle Y, \le \rangle $ ... Non-strict partially ordered set | | ||
| - | | @#55CCEE: context | @#55CCEE: $ s:B\to Y $ | | + | | @#55CCEE: context | @#55CCEE: $ r:B\to Y $ | |
| - | | @#FF9944: definition | @#FF9944: $ O_s := \{\beta\in B\mid \forall(b\in X).\,s(\beta)\le{s(b)}\}$ | | + | | @#FF9944: definition | @#FF9944: $ O_r := \{\beta\in B\mid \forall(b\in B).\,r(\beta)\le{r(b)}\}$ | |
| ----- | ----- | ||
| - | If ${\mathrm{min}(s)}\subseteq B$ denote the minimum values of $s$, then | + | If ${\mathrm{min}(r)}\subseteq Y$ denote the minimum values of $r$, then |
| - | $O_s = s^{-1}({\mathrm{min}(s)})$ | + | $O_r = r^{-1}({\mathrm{min}(r)})$ |
| - | with $s^{-1}:{\mathcal P}Y\to{\mathcal P}B$. | + | with $r^{-1}:{\mathcal P}Y\to{\mathcal P}B$. |
| Compare with [[Solution set]]. | Compare with [[Solution set]]. | ||
| - | |||
| - | == Example == | ||
| - | For | ||
| - | |||
| - | $s:{\mathbb R}\to{\mathbb R}$ | ||
| - | |||
| - | $s(x):=(x-7)^2$ | ||
| - | |||
| - | we get | ||
| - | |||
| - | $O_s=\{7\}$ | ||
| === Parametrized regression === | === Parametrized regression === | ||
| Line 33: | Line 22: | ||
| where $x_0$ somehow depends on $y_0$. | where $x_0$ somehow depends on $y_0$. | ||
| - | Use $B$-family of fit functions (hypothesis) | + | Use $B$-family of fit functions |
| $f:B\to(X\to Y)$ | $f:B\to(X\to Y)$ | ||
| - | and find from this set find the optimal fit (given by optimal $\beta\in B$) w.r.t. loss function $V:Y\to Y$ by optimizing | + | (the indexed subspace of $X\to Y$ is called hypotheses space) |
| + | |||
| + | and find from this set find the optimal fit (given by optimal $\beta\in B$) w.r.t. loss function | ||
| + | |||
| + | $V:Y\times Y\to Y$ | ||
| + | |||
| + | by optimizing | ||
| - | $s(\beta):=V(f(\beta,x),y)$ | + | $r(\beta):=V(f(\beta,x),y)$ |
| As a remark, given a function $f$ (resp. a $\beta$), the value $V(f(\beta,x_0),y_0)$ (or a multiple thereof) is called "empirical risk" in Statistical learning theory. | As a remark, given a function $f$ (resp. a $\beta$), the value $V(f(\beta,x_0),y_0)$ (or a multiple thereof) is called "empirical risk" in Statistical learning theory. | ||
| Line 48: | Line 43: | ||
| with loss function | with loss function | ||
| - | $V(y',y)=(y'-y)^2$ | + | $V({\hat y},y)=({\hat y}-y)\cdot({\hat y}-y)$ |
| - | In practice, $x_i$ may be vectors and then $w$ is taken to be an inner product. | + | In practice, $x_i$ may be vectors and then $V$ is taken to be an inner product. |
| === Reference === | === Reference === | ||
