Multidimensional Taylor's Series

Drivabletree · 2015-11-08T16:19:26+00:00

Hi,

Can someone help me out an explain how you expand the f(x) for this series, i cant work out why the 3rd term is arranged like it is? Could anyone write the 4th term and explain how you got it?

Thanks

flunzi · 2015-11-08T23:05:02+00:00

(Warning, this post is long and may seem complicated, just read the last part if it's too much :-).

The 4th term is hard to write in this vector notation.

2 ways to look at it:

(Option 1 - geometric intuition)

The n-th derivative of f at a is, by definition of derivatives in higher dimensions (Frechet-derivative), a map/function that takes n vectors as arguments.

For example:

f(a) is the 0-th derivative of f at a, and is a constant vector (it has no argument - if this confuses you, read on).

Thus, the 1st term is written as f(a).

f'(a) is the 1st derivative of f at a, is represented by a row vector (n components - see slide) and takes one vector argument of the form (x-a). Evaluating the 1st derivative is done by multiplying the row vector (gradient of f evaluated at a) with the argument column vector.

You could write the 2nd term as 1/1! f'(a)[x-a] - the [] brackets meaning "(x-a) is an argument of the map f'(a)".

f''(a) is the 2nd derivative of f at a, is represented by the "2-dimensional" n x n matrix H_f(a) (Hessian matrix of f at a) and takes 2 vectors of the form (x-a) as arguments. Evaluating the 2nd derivative is done by multiplying the Hessian "from both sides" with both argument vectors -row vector (1xn) times matrix (nxn) times column vector (nx1).

You could write the 3rd term as 1/2! f''(a)[x-a,x-a] or as 1/2! H(x=a) [x-a,x-a] - meaning that the map f''(a)=H(x=a) has 2 vector arguments of the form (x-a) and (x-a).

f'''(a) would be the third derivative, and takes 3 vectors (x-a) as arguments. If you represented f'''(a) by something, it would be a n by n by n "cube matrix" ("3 dimensions"), which is multiplied with (x-a) "from all 3 sides" to evaluate it.

So the 4th term would look something like 1/3! f'''(a)[x-a,x-a,x-a].

As you can see, writing down representations of higher derivatives needs more dimensions than your sheet of paper has, which is stupid (and why your formula stops being explicitly written down at the second derivative step).

But you can imagine the 3rd derivative as a 3-dimensional "cubic matrix" with n*n*n entries that has 3 indices, just like df/dx has one index d/dx_i f (one index for each partial derivative) and n entries , and the Hessian has 2 indices d/dx_i d/dx_j f (2 indices for any combination of 2 partial derivatives) and n² entries.

The 4th derivative would be represented as a 4-dimensional cube matrix thingie with n⁴ entries of the form

d/dx_i d/dx_j d/dx_k d/dx_l f (where i,j,k,l are any indices from 1 to n).

Writing this down explicitly is either pretty much abstract -

f''''(a)[x-a,x-a,x-a,x-a]

or can't be done unless you have a 4-dimensional piece of paper :-)

f is a function that takes a vector and has a scalar ("zero dimensional" on your piece of paper - no width or height) result. You can represent the n-th derivative of f at a as an n-dimensional object, and every time you multipy it "from one side" with one vector (x-a), you reduce the dimension by one, until you end up being "zero-dimensional" (a real number).

Which makes sense, because in the end the part f(x) left of the "=" is a real number (a scalar), so everything that is added on the right side must be scalars too (otherwise the formula makes no sense).

(Option 2 - these are just sums, don't get confused)

Keep in mind that matrix vector notation is just a clever way to write sums.

For example, multiplying a row vector u with a column vector v is just short for writing the sum

sum_i u_i v_i

where u_i, v_i are the i-th components of the vectors u and v, and you just multiply them and then add up over all i from 1 to n.

In the same way, f'(a) * (x-a) is just the sum (see picture - i'm using different notations, the first being the one from your slide, the last being the sum)

What is a tensor?
Asked by: Kelly Garmond

Answer

Tensors, defined mathematically, are simply arrays of numbers, or functions, that transform according to certain rules under a change of coordinates. In physics, tensors characterize the properties of a physical system, as is best illustrated by giving some examples (below).

A tensor may be defined at a single point or collection of isolated points of space (or space-time), or it may be defined over a continuum of points. In the latter case, the elements of the tensor are functions of position and the tensor forms what is called a tensor field. This just means that the tensor is defined at every point within a region of space (or space-time), rather than just at a point, or collection of isolated points.

A tensor may consist of a single number, in which case it is referred to as a tensor of order zero, or simply a scalar. For reasons which will become apparent, a scalar may be thought of as an array of dimension zero (same as the order of the tensor).

An example of a scalar would be the mass of a particle or object. An example of a scalar field would be the density of a fluid as a function of position. A second example of a scalar field would be the value of the gravitational potential energy as a function of position. Note that both of these are single numbers (functions) that vary continuously from point-to-point, thereby defining a scalar field.

The next most complicated tensor is the tensor of order one, otherwise known as a vector. Just as tensors of any order, it may be defined at a point, or points, or it may vary continuously from point-to-point, thereby defining a vector field. In ordinary three dimensional space, a vector has three components (contains three numbers, or three functions of position). In four dimensional space-time, a vector has four components. And, generally, in an n-dimensional space, a vector (tensor of order one) has n components. A vector may be thought of as an array of dimension one. This is because the components of a vector can be visualized as being written in a column or along a line, which is one dimensional.

An example of a vector field is provided by the description of an electric field in space. The electric field at any point requires more than one number to characterize because it has both a magnitude (strength) and it acts along a definite direction, something not shared with a scalar, such as mass. Generally, both the magnitude and the direction of the field vary from point-to-point.

As might be suspected, tensors can be defined to all orders. Next above a vector are tensors of order 2, which are often referred to as matrices. As might also be guessed, the components of a second order tensor can be written as a two dimensional array.. Just as vectors represent physical properties more complex than scalars, so too matrices represent physical properties yet more complex than can be handled by vectors.

An example of a second order tensor is the so-called inertia matrix (or tensor) of an object. For three dimensional objects, it is a 3 x 3 = 9 element array that characterizes the behavior of a rotating body. As is well known to anyone who has played with a toy gyroscope, the response of a gyroscope to a force along a particular direction (described by a vector), is generally re-orientation along some other direction different from that of the applied force or torque. Thus, rotation must be characterized by a mathematical entity more complex than either a scalar or a vector; namely, a tensor of order two.

There are yet more complex phenomena that require tensors of even higher order. For example, in Einstein's General Theory of Relativity, the curvature of space-time, which gives rise to gravity, is described by the so-called Riemann curvature tensor, which is a tensor of order four. Since it is defined in space-time, which is four dimensional, the Riemann curvature tensor can be represented as a four dimensional array (because the order of the tensor is four), with four components (because space-time is four dimensional) along each edge. That is, in this case, the Riemann curvature tensor has 4 x 4 x 4 x 4 = 256 components! [Fortunately, it turns out that only 20 of these components are mathematically independent of each other, vastly simplifying the solution of Einstein's equations].

Finally, to return to the comment that tensors transform according to certain rules under a change of coordinates, it should be remarked that other mathematical entities occur in physics that, like tensors, generally consist of multi-dimensional arrays of numbers, or functions, but that are NOT tensors. Most noteworthy are objects called spinors. Spinors differ from tensors in how the values of their elements change under coordinate transformations. For example, the values of the components of all tensors, regardless of order, return to their original values under a 360-degree rotation of the coordinate system in which the components are described. By contrast, the components of spinors change sign under a 360-degree rotation, and do not return to their original values until the describing coordinate system has been rotated through two full rotations = 720-degrees!
Answered by: Warren Davis, Ph.D., President, Davis Associates, Inc., Newton, MA USA

https://latex.codecogs.com/gif.download?%28%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x%7D%20f%20%7C_%7Bx%3Da%7D%20%29%5Ccdot%20%28x-a%29%20%3D%20f%27%28a%29%5Bx-a%5D%20%3D%20%5Cnabla%20f%28a%29%20%5Ccdot%20%28x-a%29%20%3D%20%5Csum_%7Bi%3D1%7D%5En%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20x_i%7D%28a%29%28x-a%29_i

where (x-a)_i is the i-th component of the vector (x-a).

In the same way, (x-a)' * H * (x-a) (row vector * Hessian matrix * column vector) is just a double sum

https://latex.codecogs.com/gif.download?%28x-a%29%27%20%5Ccdot%20%28%20%5Cfrac%7B%5Cpartial%5E2%20f%7D%7B%5Cpartial%20x%5E2%7D%20%7C_%7Bx%3Da%7D%20%29%20%5Ccdot%20%28x-a%29%20%3D%20f%27%27%28a%29%5Bx-a%2Cx-a%5D%20%3D%20%5Csum_%7Bi%3D1%7D%5En%20%5Csum_%7Bj%3D1%7D%5En%20%28x-a%29_i%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_i%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_j%7D%20f%28a%29%20%28x-a%29_j

If this confuses you, this is just how vector times matrix times vector multiplication is defined - ends up being sums.

So the third derivative of f at a, evaluated at the 3 arguments (x-a), (x-a) and (x-a) would just be a triple sum. This would be the 4th term you asked for. Here's a picture:

https://latex.codecogs.com/gif.download?%5Csum_%7Bi%3D1%7D%5En%20%5Csum_%7Bj%3D1%7D%5En%20%5Csum_%7Bk%3D1%7D%5En%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_i%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_j%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_k%7D%20f%20%28a%29%20%28x-a%29_i%20%28x-a%29_j%20%28x-a%29_k

You can simplify the sum notation if you use multiindices - see here

https://en.wikipedia.org/wiki/Multi-index_notation

but if this isn't part of your lecture, don't waste too much thought on that.

The most important thing to know:

the Taylor formula on your slide is just a notation for sums (single sum for gradient * vector, double sum for row vector * matrix * vector).
this does not look like a math lecture, so you will probably never see higher derivatives than gradient (vector) and Hessian (matrix), so don't worry about that. Just know how to apply the formula as it is written on the slide to concrete examples (so compute the gradient and the hessian), and you should be good. (Maybe you don't even have to do that).

For completeness, this is how the Taylor formula from your slide is written explicitly as sums, and this is just what it means, nothing more.

https://latex.codecogs.com/gif.download?f%28x%29%20%3D%20f%28a%29%20+%20%28%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x%7D%20f%20%7C_%7Bx%3Da%7D%20%29%5Ccdot%20%28x-a%29%20+%20%28x-a%29%27%20%5Ccdot%20%28%20%5Cfrac%7B%5Cpartial%5E2%20f%7D%7B%5Cpartial%20x%5E2%7D%20%7C_%7Bx%3Da%7D%20%29%20%5Ccdot%20%28x-a%29%20+%20%5Cldots%20%3D%20%5Cnewline%20%3D%20f%28a%29%20+%20%5Cnewline%20+%20%5Csum_%7Bi%3D1%7D%5En%5Cfrac%7B%5Cpartial%20f%7D%7B%5Cpartial%20x_i%7D%7C_%7Bx%3Da%7D%28x-a%29_i%20+%20%5Cnewline%20+%20%5Csum_%7Bi%3D1%7D%5En%20%5Csum_%7Bj%3D1%7D%5En%20%28x-a%29_i%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_i%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_j%7D%20f%7C_%7Bx%3Da%7D%20%28x-a%29_j%20+%20%5Cldots

and the next (4th) term would be

https://latex.codecogs.com/gif.download?%5Csum_%7Bi%3D1%7D%5En%20%5Csum_%7Bj%3D1%7D%5En%20%5Csum_%7Bk%3D1%7D%5En%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_i%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_j%7D%20%5Cfrac%7B%5Cpartial%7D%7B%5Cpartial%20x_k%7D%20f%20%28a%29%20%28x-a%29_i%20%28x-a%29_j%20%28x-a%29_k

Monday, February 29, 2016

Answer

Sunday, February 28, 2016

[PDF]4

Saturday, February 27, 2016