Technology Sharing

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

1. Sigmoid activation function:

The Sigmoid function is an important activation function for building a logistic regression model, as shown in the figure below.

• twoThe goal of the classification problem is to control the output of the model within the range of [0,1].When the model output result is less than 0.5, the default prediction result is 0; when the model output result is greater than 0.5, the default prediction result is 1.
• The solution to the binary classification problem is to construct a logistic regression model f to map the input x of the binary classification problem to the input z of the Sigmoid function to calculate the output g, and then obtain the result of the logistic regression model (that is, the result of the binary classification problem) based on the range of g (whether it is greater than 0.5).
• The domain of the function ∈R, the range ∈[0,1], whenInput z＜0WhenOutput result g＜0.5,The default isThe result is 0,constituteThe first category of the binary classification problem.whenInput z＞0WhenOutput result g＞0.5,The default isThe result is 1,constituteThe second category of binary classification problems。

2. Introduction to Logistic Regression:

Logistic regression is used to solveBinary classification problemThe output of the model is limited to a certain number of classification problems (infinite for regression problems).A binary classification problem means that the model has only two output results.

In the classic example of regression problem "tumor prediction case", the tumor size feature is used to predict whether the tumor is malignant. There are only two output results: yes (1) or no (0).

At this time, it is difficult to fit the training set using a linear regression model(Linear regression solves a regression problem, while the tumor prediction case is a classification problem, or more precisely, a binary classification problem), so the idea of ​​logistic regression was proposed.

Logistic regression model (solving classification problems): input feature or feature set X and output a number between 0 and 1, where the fitting curve is constructed by the Sogmoid function. The specific construction process is as follows:

• First line explanation:Logistic regression modelfThe construction is the same as linear regression, throughInput feature set X Output prediction result fThe difference isf takes a value range of ∈[0,1]。
• Explanation of the 234th line: We have previously introduced that the output g of the Sigmoid function can solve the binary classification problem very well, so we cleverly use the Sigmoid function to build a logistic regression model f to solve the binary classification problem.Map the input feature set X to the input z of the Sigmoid function using linear regression or polynomial regression，Implement the output of the Sigmoid function，Then, the output f (0 or 1) of the logistic regression model is calculated based on whether the output of the Sigmoid function is greater than 0.5., and get the result of the two-classification problem.
• Explanation of the fifth line: By integrating the above ideas, we can get the logistic regression model f, where the input of the model is the feature set X and the output is the classification prediction result 0 or 1.
• Explanation of the sixth line: When the output result of the logistic regression model is greater than or equal to 0.5, the predicted value y^ is 1, which means that the tumor is a malignant tumor in the above example; when the output result of the logistic regression model is less than or equal to 0.5, the predicted value is 0, which means that the tumor is not a malignant tumor in the above example.

Decision Boundary

It is not difficult to see from the above that when the input z of the Sigmoid function is greater than or equal to 0, that is, the mapping z=wx+b of the feature set X to z is greater than or equal to 0, the output result of the model is 1; when the input z of the Sigmoid function is less than 0, that is, the mapping z=wx+b of the feature set X to z is less than 0, the output result of the model is 0.
This is how we can come up with the concept of a decision boundary:The equation that makes the mapping from the model input X to the Sigmoid function input z equal to 0 is called the decision boundary.

Taking the above tumor prediction model as an example, the mapping from model input X to Sigmoid function input z is z=wx+b, so the decision boundary is wx+b=0.

Let's use a graphic to illustrate the meaning of the decision boundary:

• example 1:Mapping to a linear function
  
  The figure above shows the true value of the label when the features x1 and x2 in the training set take different values. The circle represents that the classification result of the sample is 0, and the cross represents that the classification result of the sample is 1.

  The logistic regression model is shown in the figure above, where the mapping from the model input X to the Sigmoid function input z is z=w1x1+w2x2+b, and the decision boundary is w1x1+w2x2+b=0. If the model training result is w1=1, w2=1, b=-3, the decision boundary is x1+x2-3=0, and the function graph of the decision boundary is shown in the figure above.It can be seen that if the feature of the sample is on the left side of the decision boundary, the logistic regression prediction is 0, otherwise it is 1. This is the graphic meaning of the decision boundary.

• Example 2:Mapping to polynomial function
  
  The mapping from the model input X to the Sigmoid function input z is a polynomial function. The decision boundary is shown in the figure. It can be seen that after the model training is completed, the parameter values ​​are determined and the decision boundary is immediately determined.The position of the features of a sample relative to the decision boundary determines the prediction result of the sample.

4. Logistic regression model training process:

In fact, it is the same as the linear regression training process, except that the model (function) to be trained is different.

1. Training objectives:

2. Gradient descent adjustment parameters: