0.1 Partial Derivatives

Finally! We are discussing about derivatives for multivariable functions. The derivatives are actually more complicated than single-variable functions, and in this note we will discuss about partial derivatives, not directional derivatives, of multivariable functions.

First, recall that single-variable function \(f\), its derivative is defined as the following. \[ f'(x) = \lim _{h \to 0} \frac {f(x + h) - f(x)}{h} \] Geometrically interpreting this, it can be considered as the slope of the tangent line at the point. Another way of interpreting that was instantaneous rate of change that the point.

For multivariable functions, it should be similar. But how do we determine which tangent line to use? If we think about it, it is actually not very easy to determine because there are many tangent lines at a specific point. Also, if we think of it as instantaneous rate of change, the direction of the change can give different values. This is where directional derivatives comes in, but in this section, we will be discussing about a more simple derivative that we can think of from multivariable functions.

For functions with two variables, we don’t really have to care about direction if we slice a function and consider the derivative based on that slicing line. That slicing line can be a line parallel to \(x\) or \(y\) axis. If we consider them as special derivatives, we can define partial derivatives as the following.

Definition 0.1.1

Given a function \(f(x, y)\), its partial derivative with respect to \(\mathbf {x}\) at point \((x_0, y_0)\) is defined as the following. \[ f_x (x_0, y_0) = \lim _{h \to 0} \frac {f(x_0 + h, y_0) - f(x_0, y_0)}{h} \]

Similarly for partial derivative with respect to \(y\), we can write it as \(f_y (x_0, y_0) = \lim _{h \to 0} \frac {f(x_0, y_0 + h) - f(x_0, y_0)}{h}\). This is much easier because by slicing and thinking only about that slice, it becomes ordinary derivative of a single-variable function. Another way of thinking partial derivatives is we are simply putting the other variable as a constant and finding the derivative with the remaining single-variable function. Here is an example.

Exercise 0.1.2

Given \(f(x, y) = x^2 + 2x + y^2\), Find \(f_x\) and \(f_y\) at \((1, 2)\).

Solution.

Consider the following equation for \(f_x(1, 2)\). \begin{align*} f_x (1, 2) &= \lim _{h \to 0} \frac {f(1 + h, 2) - f(1, 2)}{h} \\ &= \lim _{h \to 0} \frac {h^2 + 2h + 1 + 2h + 2 + 4 - 7}{h} \\ &= \lim _{h \to 0} \frac {h^2 + 4h}{h} = \lim _{h \to 0} h + 4 = 4 \end{align*}

Therefore, \(f_x(1, 2) = 4\).

For \(f_y(1, 2)\) let’s take a different approach. Considering \(x\) as a constant and finding the derivative with respect to \(y\), we get \(f_y = 2y\). Therefore, \(f_y(1, 2) = 4\).

Before moving on to higher order partial derivatives, here is another very popular way of writing partial derivatives. \[ f_x = \frac {\partial f}{\partial x} \quad \text {and}\quad f_y = \frac {\partial f}{\partial y} \] Now how do we take higher order partial derivative of \(f\)?

First, notice that \(f_x\) and \(f_y\) are the first-order partial derivatives of \(f\). Using this, we can find the second order. \begin{align*} f_x &\rightarrow (f_x)_x,\ (f_x)_y \\ f_y &\rightarrow (f_y)_x,\ (f_y)_y \end{align*}

Note that we usually suppress the parenthesis and just simply write \(f_{xx}, f_{xy}, f_{yx}, f_{yy}\). Using these, we can also find the third-order partial derivatives of \(f\). \begin{align*} f_{xx} &\rightarrow f_{xxx},\ f_{xxy} \\ f_{xy} &\rightarrow f_{xyx},\ f_{xyy} \\ f_{yx} &\rightarrow f_{yxx},\ f_{yxy} \\ f_{yy} &\rightarrow f_{yyx},\ f_{yyy} \end{align*}

As long as the resulting partial derivatives are differentiable, we can continue this process! Notice that if we use the other notation, then the order in which write the partial derivatives are flipped. Here is what I mean. \[ f_{xyy} = \frac {\partial }{\partial y} \left ( \frac {\partial }{\partial y} \left ( \frac {\partial f}{\partial x} \right ) \right ) = \frac {\partial ^3 f}{\partial y\, \partial y\, \partial x} \] With the notation involving the partial symbol, we read it from right to left. Here is an example.

Exercise 0.1.3

Given \(f(x, y) = x^2y^3 + 2x + 3y^2\), compute \(f_{xyy}\).

Solution.

First, notice that we need to find the partial derivative of \(f\) with respect to \(x\). \[ \frac {\partial f}{\partial x} = 2xy^3 + 2 \] Continuing, we can find the following. \[ \frac {\partial ^2}{\partial y^2} (2xy^3 + 2) = \frac {\partial }{\partial y} 6xy^2 = 12xy \] Therefore, \(\frac {\partial ^3 f}{\partial y^2 \partial x} = 12xy\).

Continuing, we can consider a very important theorem.

Theorem 0.1.4: Clairaut’s Theorem

For a function \(f\) of two variables, if \(f_{xy}\) and \(f_{yx}\) are continuous on an open set \(A\), then \(f_{xy} = f_{yx}\) for all points in \(A\).

(Video Proof)

Proof.

The proof can be done by applying MVT two times for two different cases. First, define \(\Delta h = [f(a + h, b + h) - f(a + h, b)] - [f(a, b + h) - f(a, b)]\).

For the first case, define \(g(x) = f(x, b + h) - f(x, b)\). By construction, it is evident that \(g\) is a single-variable function and \(\Delta h = g(a + h) - g(a)\). By MVT, there exists \(c \in (a, a + h)\) such that \(g'(c) = \frac {g(a + h) - g(a)}{a + h - a}\), or \(h g'(c) = g(a + h) - g(a)\). In other words, \(\Delta h = hg'(c)\) for some constant \(h\) and \(c\). Continuing, notice that \(g'(c) = f_x f(c, b + h) - f_x f(c, b)\) by definition. Using MVT again, there exists \(d \in (b, b + h)\) such that \(h f_{xy} (c, d) = g'(c)\) since \(c\) is a constant. In other words, \(\Delta h\) can be written as \(h^2 f_{xy}\) and the following equation holds. \[ \lim _{h \to 0} \frac {\Delta h}{h^2} = \lim _{(c, d) \to (a, b)} f_{xy} (c, d) = f_{xy} (a, b) \] For the second case, let \(g(y) = f(a + h, y) - f(a, y)\). Notice that \(\Delta h = [f(a + h, b + h) - f(a, b + h)] - [f(a + h, b) - f(a, b)] = g(b + h) - g(b)\). Similarly by MVT, there exists \(c \in (b, b + h)\) such that \(h g'(c) = g(b + h) - g(b) = \Delta h\). Moreover, because \(g'(c) = f_y (a + h, c) - f_y (a, c)\), we MVT can be used again to show that there exists \(d \in (a, a + h)\) such that \(h f_{yx} (d, c) = g'(c)\). Therefore, \(\Delta h\) can be rewritten as \(h^2 f_{yx} (d, c)\), and the following equation holds. \[ \lim _{h \to 0} \frac {\Delta h}{h^2} = \lim _{(d, c) \to (a, b)} f_{yx} (d, c) = f_{yx} (a, b) \] Therefore, \(f_{xy} (a, b) = f_{yx} (a, b)\) for all points \((a, b)\) in \(\dom (f)\) and the theorem holds.

In fact, this theorem is not limited to functions of two variables at all! We can simply let \(g(x_i, x_j) = f(x_1, \ldots , x_i, \ldots , x_j, \ldots , x_n)\) and notice that \(g_{x_i x_j} = f_{x_i x_j}\) and \(g_{x_j x_i} = f_{x_j x_i}\). Because \(g_{x_i x_j} = g_{x_j x_i}\), \(f_{x_i x_j} = f_{x_j x_i}\) for function \(f\) of \(n\) variables where \(n > 2\).

Notice that this theorem implies that we can switch the order of writing \(x,y\) for higher order partial derivatives. Meaning, all it matters is the number of \(x\) and \(y\) in the subscript, not their orders when taking higher order partials.

Finally, for a function \(f\) of \(n\) variables, our discussion can be generalized as the following. \[ f_{x_i} (x_1, \ldots , x_n) = \lim _{h \to 0} \frac { f(x_1, \ldots , x_i + h, \ldots , x_n - f(x_1, \ldots , x_n) }{h} \]

Now that we can take partial derivatives of a multivariable function, how do we define differentiability?

0.1.1 Differentiability

In single variable calculus, we said a function is differentiable at a given domain if it is differentiable at all points in the domain. Now how do we apply similar notion for multivariable functions? Notice that in a sense, partial derivatives of a multivariable function is different from derivatives of single variable functions. Therefore, although differentiability of a multivariable function implies the existence of partial derivatives, we need a different way to define differentiability for multivariable functions.

Although partial derivatives provide very limited information about the function at the point, the fact that differentiability implies continuity at a point remains. Also before moving on to differentiability for multivariable functions, let’s investigate a little more on the differentiability tells us about a function.

Let \(f\) be a single variable function, \(I\) be an open interval around \(x_0\), \(\epsilon \) be a function where \(\epsilon \to 0\) as \(x \to x_0\), and \(m\) be a number. \(f\) is differentiable at the point \(x_0\) if and only if the following equation holds. \[ f(x) - f(x_0) = m(x - x_0) + \epsilon (x) (x - x_0) \]

We are going to first show this because this leads us to an interesting finding. First, assume that \(f\) is differentiable and the following equation holds. \[ \lim _{x \to x_0} \frac {f(x) - f(x_0)}{x - x_0} = m \] With this, define \(\epsilon (x)\) as the following piece-wise function. \[ \epsilon (x) = \begin {cases} \frac {f(x) - f(x_0)}{x - x_0} - m & (x \neq x_0) \\ 0 & (x = x_0) \end {cases} \] Notice that by construction, \(\epsilon (x)\) approaches zero as \(x \to x_0\). Rearranging the equation, we obtain the following. \[ f(x) - f(x_0) = m(x - x_0) + \epsilon (x) (x - x_0) \] Now to show the converse, we can rearrange the equation as \[ \frac {f(x) - f(x_0)}{x - x_0} = m + \epsilon (x) \] for \(x \neq x_0\) and we can see that the statement holds. Now this fact leads us to an interesting interpretation of differentiability. Actually, the interpretation that we are going to see now can be quite familiar, but it gets very interesting when we extend it to multivariable functions.

Let \(L(x) = m(x - x_0) + f(x_0)\). Note that \(L(x)\) is a linear function that passes through \((x_0, f(x_0))\). Rewriting the statement we just proved, we can see that a function is differentiable if and only if there exists a line \(L\) that passes through the point \((x_0, f(x_0))\) and that satisfies the following. \[ \lim _{x \to x_0} \frac {f(x) - L(x)}{x - x_0} = 0 \] Now this unique \(L\) is just a tangent line with slope \(m\), but I think this is an interesting way to algebraically show the connection between the existence of the good approximation and differentiability.

Now how do we expand this to functions with more variables? For functions of two variables, we can define differentiability as the following.

Definition 0.1.5

Given a function \(f\) that is defined in a open disk \(D\) around a point \((x_0, y_0)\), \(f\) is differentiable if and only if there exists some function \(\epsilon _1(x, y)\) and \(\epsilon _2(x, y)\) such that \(\lim _{(x, y) \to (x_0, y_0)} \epsilon _1 = \lim _{(x, y) \to (x_0, y_0)} \epsilon _2 = 0\) and the following equation holds. \begin{align*} f(x, y) - f(x_0, y_0) &= f_x(x_0, y_0) (x - x_0) + f_y(x_0, y_0) (y - y_0) \\ &\qquad \qquad \ + \epsilon _1(x, y) (x - x_0) + \epsilon _2(x, y) (y - y_0) \end{align*}

Just as we had the unique tangent line, this time, \(f_x(x_0, y_0) (x - x_0) + f_y(x_0, y_0) (y - y_0)\) can be interpreted as the tangent plane that is a good approximation of \(f\) near the point \((x_0, y_0)\). From the equation, we can also notice that \[ \lim _{(x, y) \to (x_0, y_0)} f(x, y) = f(x_0, y_0) \] and that differentiability implies continuity.

Now before wrapping up this section, here is an important theorem!

Theorem 0.1.6

Let \(f\) be a function whose first-order partials are defined on an open disk around \((x_0, y_0)\). \(f\) is differentiable at \((x_0, y_0)\) if the defined partials are continuous at the point \((x_0, y_0)\).

The proof for this theorem is quite long, and may not fit to the scope of this note. So let’s skip the proof for now, but maybe I can add it in the future!

This theorem also tells that under the same condition, we can see that \(f\) is continuous at \((x_0, y_0)\) since differentiability implies continuity. Now let’s discuss chain rule before we conclude this section!

0.1.2 The Chain Rule