Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Multidimensional features

Variables and terms

formula

$\vec{w}$ = [w₁ w₂ w₃ …]
$\vec{x}$ = [x₁ x₂ x₃ …]

$f_{\vec{w},b}(\vec{x})=\vec{w}\ast \vec{x}+b=w_1{x}_1+w_2{x}_2+\dots +w_n{x}_n+b$

Multiple Linear Regression

import numpy
f = np.dot(w, x) + b
1
2

Note: When n is large, it is very fast (parallel processing)

Normal equation method

1. Greater than 1000, low efficiency
2. It cannot be generalized to other algorithms such as logistic regression, neural networks or others.
3. No iteration

$w_n=w_n-\alpha \frac{1}{m}\sum _{i=1}^m{f}_{\vec{w},b}({\vec{x}}^{(i)}-y^{(i)})x_n^{(i)}$

$b=b-\alpha \frac{1}{m}\sum _{i=1}^m(f_{\vec{w},b}({\vec{x}}^{(i)}-y^{(i)})$

The weights corresponding to independent variables with larger ranges tend to be smaller, and the weights corresponding to independent variables with smaller ranges tend to be larger.

Mean normalization

Divide by the maximum value of the range to get the weight and the independent variable [0, 1]

Horizontal axis:$x_1=\frac{x_1-{\mu }_1}{2000-300}$ Y-axis:$x_2=\frac{x_2-{\mu }_2}{5-0}$

$-0.18\le x_1\le 0.82$ $-0.46\le x_2\le 0.54$

Z-score normalization

$300\le x_1\le 2000$ $0\le x_2\le 5$

$x1=\frac{x_1-{\mu }_1}{{\sigma }_1}$ $-0.67\le x_1\le 3.1$

By scaling, we try to keep the values ​​of all features in a similar range, so that the impact of their changes on the predicted values ​​is close to (-3,3)

If the cost function J becomes larger, then the step size (learning rate) is inappropriate or the code is wrong.

Note: The number of iterations varies from machine to machine.

In addition to drawing curves to determine the extent of the iteration, you can also use automatic convergence testing
Let ε be equal to$10^{-3}$, if the decrease of J is less than this small number, it is considered to have converged.

Set an appropriate learning rate

1. When testing, you can set a very small value to see if J decreases.
2. The learning rate should not be too large or too small during iteration
3. When testing, each time * 3, choose the largest possible learning rate, or a slightly smaller value than the reasonable value

Feature engineering

Create feature engineering through transformation or combination to provide more options

$f_{\vec{w},b}(\vec{x})=w_1{x}_1+w_2{x}_2+w_3{x}_3+b$

Note: Polynomial regression can be used for both linear and nonlinear fits.