Optimization, as one of the fundamental processes in several machine learning algorithms, is dependent on the leveraging of derivatives in order to determine in which fashion to go about updating a model’s parameters values, to maximize or minimize an objective function.

This blog post by AICoreSpot will serve as a tutorial and continue exploring the different strategies through which we can identify the derivatives of functions. Specifically, we will be delving into the product, power and quotient rules, which we can leverage to arrive to the derivatives of functions quicker than if we had to identify each derivative from first principles. Therefore, for functions that are particularly challenging, keeping these rules at hand to identify their derivatives will become more and more critical.

In this tutorial, you will find out about the product, power and quotient rules to identify the derivative of functions.

After finishing this tutorial, you will be aware of:

• The power rule to adhere to when identifying the derivative of a variable base, raised to a fixed power.
• How the product rules facilitates us to identify the derivative of a function that is defined as the product of another two (or more functions)
• How to the quotient rule enables us to identify the derivative of a function that is ratio of two differentiable functions.

Let’s begin:

This tutorial consists of three parts:

• The Power Rule
• The Product Rule
• The Quotient Rule

The Power Rule

If we possess a variable base raised to a fixed power, the rule to adhere to in order to identify its derivative is to bring down the power in front of the variable base, and then subtract the power by 1.

For instance, if we possess the function f(x) = x squared, which we would desire to identify the derivative, we first bring down 2 in front of x and the subtract the power by 1.

f(x) = x squared

f (x) = 2x

In the pursuit of comprehending better where this rule has its origins, let’s take the longer path and identify the derivative of f(x) by beginning from the definition of a derivative.

Here, we replace for f(x) = x squared and then proceed to simplify the expression.

As h approaches a value of 0, then this limit approaches 2x, which tallies with the outcome that we have received prior leveraging the power rule.

If applied to f(x) = x, the power rule provides us a value of 1. This is due to the fact that when we bring a value of 1 in front of x, and then reduce the power by 1, what we are left with is a value of 0 in the exponent. Since x to the power of 0 is equal to 1, then f(x) = (1) (x to the power of 0) = 1.

The best way to comprehend this derivative is to know that f(x) = x is a line that fits the form y = mx +b owing to the fact that f(x) = x is the same as f(x) = 1x + 0 (or y = 1x + 0). The slope (m) of this line is 1, therefore, the derivative is equivalent to 1. Or you can just memorize that the derivative of x is 1. But if you don’t remember both of these concepts, you can always leverage the power rule.

The power rule can be used on any power, regardless of whether it is positive, negative, or a fraction. We can also apply it to radical functions by first expressing their exponent (or power) as a fraction:

f(x) = x= x to the power of ½

f(x) = (1/2) x to the power of -1/2

The Product Rule

Let’s assume that we now possess a function, f(x) of which we like to identify the derivative, which is the product of another two functions u(x) = 2x squared and v(x) = x cubed.

f(x) = u(x) v(x) = (2x squared) (x cubed)

In order to find out how to identify the derivative of f(x), let’s first begin with identifying the derivative of the product of u(x) and v(x) directly.

(u(x) v(x)) = (2x squared) (x cubed) = (4x) (3xsquared) = 12x cubed

It is obvious that the second outcome does not tally with the initial one, and that is owing to the fact that we have not leveraged the product rule.

The product rule informs us that the derivative of the product of two functions can be identified as:

f(x) = u(x) v(x) + u(x) v(x)

We can arrive at the product rule if we work our way through application of the attributes of limits, beginning again with the definition of a derivative.

We are aware that f(x) = u(x) v(x) and therefore, we can replace for f(x) f(x+h):

At this level, our objective is to factorize the numerator into various limits that can subsequently be assessed independently. For this reason, the subtraction of terms, u(x) v(x+h) – u(x) v(x+h), shall be introduced into the numerator. Its introduction does not alter the definition of f(x) that we just gotten, but it will assist us in factorising the numerator.

The outcome expression seems complex, but, if take a closer look we realize that we possess common terms that can be factored out:

The expression can be broken down even more through application of the limit laws that enable us separate the sums and products into independent limits.

The answer to our issue has now become obvious. We can observe that the starting and final terms in the simplified expression correlate to the definition of the derivative of u(x) and v(x), which we can denote through u(x) and v(x) respectively. The second term approaches the ongoing and differentiable function, v(x) as h approaches zero, whereas the third term is u(x).

Therefore, we arrive again at the product rule:

f(x) = u(x) v(x) + u(x) v(x)

With this new utility in tow, let’s reconsider finding f(x) when u(x) = 2x squared and v(x) = x cubed.

f(x) = u(x) v(x) + u(x) v(x)

f(x) = (4x) (x cubed) + (2x squared) (3x squared) = 4x to the power of 4 + 6x to the power of 4 = 10x to the power of 4

The outcome derivative now is correctly matching with the derivative of the product, (u(x) v(x)) that we have gotten prior.

This was a fairly simplistic instance that we could have computed directly to begin with. Although, we might have more complicated issues consisting of functions that cannot be multiplied directly, to which we can simply apply the product rule. For instance:

f(x) = x squared sin x

f(x) = (x squared) (sin x) + (x squared) (sin x) = 2x sin x + x squared cos x

We can even have extension of the product rule to more than just two functions. For instance, say f(x) is currently defined as the product of three functions, u(x), v(x) and w(x):

f(x) = u(x) v(x) w(x)

We can the enforce application of the product rule as follows:

f(x) = u(x) w(x) + u(x) v(x) w(x) + u(x) v(x) w(x)

The Quotient Rule

Likewise, the quotient rule informs us how to identify the derivative of a function f(x), which is the ratio of two differentiable functions, u(x) and v(x):

We can then enforce application of the product rule on u(x) to get:

u (x) = f(x) v(x) + f(x) v(x)

Solving back for f(x) provides us:

One last step replaces for f(x) to arrive to the quotient rule:

Application of the quotient rule and simplification of the outcome expression:

From the Pythagorean identity from trigonometry, we are aware that cos squared x + sin squared x =1, therefore:

Hence, leveraging the quotient rule, we have easily identified that the derivative of tangent is the squared secant function.