08/06/04

1. Curve Fitting

Raw data usually has noise. The values of dependent variables vary even though all the independent variables are constant. Therefore, the estimation of the trend the dependent variables is needed. This process is called regression or curve fitting. The estimated equation (matrix) satisfy the raw data. However, the equation is not usually unique, and the equation or curve with a minimal deviation from all data points is desirable. This desirable best-fitting equation can be obtained by least square method which uses the minimal sum of the deviations squared from a given set of data.

2. Least Square Method

If you have a data set (x₁,y₁), (x₂, y₂),....,(x_n, y_n) and the fitting curve f(x) has the deviation d₁, d₁, .... , d_n which are caused from each data point, the least square method produces the best fitting curve with the property as follows;

        Figure 2-1. Least square error

3. Least Square Line

The least squares line method basically uses an equation f(x) = a + bx which is a line graph and describes the trend of the raw data set (x₁,y₁), (x₂, y₂),....,(x_n, y_n). The n should be greater or equal to 2 (n ≥ 2)in order to find the unknowns a and b. So the equation for the least square line is

Figure 3-1. Least square error for line

Suppose that ∏ gets zero so that the least square error has a minimum. If you get the first derivative of ∏, the equation will be as follows;

    
Figure 3-2. Derivatives of least square error in least square line

From the Figure 4-2, we can compute the two unknowns by the process following;

    
Figure 3-3. Computation of unknown a and b in least square line

4. Least Square Parabola

The least squares line method uses an equation f(x) = a + bx + cx² which is a parabola graph. The n should be greater or equal to 3 (n ≥ 3)in order to find the unknowns a, b, and c. When you get the first derivatives of ∏ in parabola, you will have

Figure 4-1. Least square error in parabola

We can get the unknown a, b, and c in the similar way we used for the line.

5. Least Square Method in m-th degree polynomials

Now, we can think of the least square method in m-th degree polynomial. If you desire to know the m +1 number unknowns in m-th degree polynomials, you can simply follow the process of the following. Remember that the number of equations (=n) should be greater or equal to m +1 (n ≥ m +1).

1. Equation for m-th degree polynomial

Figure 5-1. Equation for m-th degree polynomial

2. Equation for least square error

Figure 5-2. Equation for least square error in m-th degree polynomial

3. Computation of unknown coefficient a⁰, a¹, a², a³, ....., a^m
(1) Get the first derivatives of ∏ in terms of all the unknown variables

Figure 5-3. First derivatives of ∏ in Figure 6-2

(2) Expand the equations in Figure 6-3

Figure 5-4. Equations rearranged from Figure 6-3

(3) Solve the equations in Figure 6-4 and find the unknowns (a⁰, a¹, a², a³, ....., a^m) by using the procedure as follows;

    X A = Y
    X^T X A = X^T Y
    A = (X^T X)^-1 X^T Y    [1]

where X^T is the transpose matrix of X and X^-1 is the inverse matrix of X.
Therefore the final equation of [1] can be expressed as the following matrices

Figure 5-5. Computations of unknowns (a⁰, a¹, a², a³, ....., a^m) using matrices

6. Multiple Regression Least Square Method

The term, Multiple regression is originated from multiple number of independent variables (control parameters), which means that the dependent variable is changed by more than one independent variable. The examples of fitting equations are as follows;

     Two independent variable   : Z = aA + bB + c           
                  (where Z : dependent variable;  A, B :  independent variables; a, b, c : constants)
     Three independent variable : Z = aA + bB + cC + d
                  (where Z : dependent variable;  A, B, C :  independent variables; a, b, c, d : constants)
                                            :
                                            :

Let's think about the multiple regression with two independent variable to simplify the situation. The least square error will be

Figure 6-1. Equation for least square error in multiple regression

The first derivatives of ∏ in terms of a and b will be

Figure 6-2. First derivatives of ∏

The equations expended from Figure 6-2 will be

Figure 6-3. Equations rearranged from Figure 6-2

Computation of the unknown constants using  A = (X^T X)^-1 X^T Y   [1] and matrices will be

Figure 6-4. Computations of unknowns (a, b, c) using matrices

7. Applications

    Figure 7-1. Marker set

Figure 7-1. Where x_t, y_t, z_t are the axes of local reference system of thigh, S1 – S5 are the markers on the thigh, NJC is the knee joint center, X_c is the         vector from the origin of thigh reference system to the knee joint center, X1_i and X2_i are vectors from the origin of thigh to the markers S1 to S5 at the position P1 and P2, respectively, and X1’_i and X2’_i are vectors from the knee joint center to the markers S1 to S5 at the position P1 and P2, respectively.

    The vector X_c which is from the origin of local referenced system of thigh to the knee joint center can be calculated using the least square method as follows,

        Figure 7-2.vector X_c which is from the origin of local referenced system of thigh to the knee joint center