This is an old revision of the document!
Optimization set
Set
context | $ B $ |
context | $ \langle Y, \le \rangle $ … Non-strict partially ordered set |
context | $ s:B\to Y $ |
definition | $ O_s := \{\beta\in B\mid \forall(b\in X).\,s(\beta)\le{s(b)}\}$ |
If ${\mathrm{min}(s)}\subseteq B$ denote the minimum values of $s$, then
$O_s = s^{-1}({\mathrm{min}(s)})$
with $s^{-1}:{\mathcal P}Y\to{\mathcal P}B$.
Compare with Solution set.
Example
For
$s:{\mathbb R}\to{\mathbb R}$
$s(x):=(x-7)^2$
we get
$O_s=\{7\}$
Parametrized regression
Consider a test set
$\langle x_0,y_0\rangle \in X\times Y$,
where $x_0$ somehow depends on $y_0$.
Use $B$-family of fit functions
$f:B\to(X\to Y)$
(the indexed subspace of $X\to Y$ is called hypotheses space)
and find from this set find the optimal fit (given by optimal $\beta\in B$) w.r.t. loss function $V:Y\to Y$ by optimizing
$s(\beta):=V(f(\beta,x),y)$
As a remark, given a function $f$ (resp. a $\beta$), the value $V(f(\beta,x_0),y_0)$ (or a multiple thereof) is called “empirical risk” in Statistical learning theory.
Linear regression w.r.t. least square
$f(\beta,x):=\beta_0+\sum_{i=1}^N\beta_ix_i$
with loss function
$V(y',y)=(y'-y)^2$
In practice, $x_i$ may be vectors and then $w$ is taken to be an inner product.