Inner Products of Functions

FS02: Inner Products of Functions

FS01 showed that a complicated periodic waveform can be assembled from harmonics. The next question is more precise:

• How can we measure how much of one candidate harmonic is contained in a function?

The answer begins with ordinary vector geometry.

For a vector, the dot product measures alignment and extracts components. Fourier analysis uses the same geometric idea, but the “vectors” are functions and the dot product becomes an integral.

The central definition of this lecture will be

(1)

This lecture develops Equation (1) slowly rather than treating it as a mysterious rule. We will build it from:

• ordinary vector dot products,
• orthogonality,
• vector projection,
• the passage from finite sums to integrals,
• function norms and normalization,
• projection of one function onto another,
• residual orthogonality,
• weighted inner products.

The trigonometric orthogonality identities themselves are postponed to FS03. By the end of FS02, however, the reason Fourier coefficients are projection coefficients should already be clear.

1 Review: the dot product of ordinary vectors

Consider two vectors in three-dimensional Euclidean space:

(2)

Their dot product is

(3)

This operation takes two vectors and returns one scalar.

For example, let

(4)

Then

a ⋅ b	= (2)(4) + (−1)(5) + (3)(−2)	(5)
	= 8 − 5 − 6	(6)
	= −3.	(7)

The dot product contains geometric information. It can be written as

(8)

where 𝜃 is the angle between the vectors.

Thus:

• a positive dot product indicates an acute angle,
• a negative dot product indicates an obtuse angle,
• a zero dot product indicates a right angle.

2 Orthogonal vectors

Two nonzero vectors are orthogonal when

(9)

From Equation (8),

(10)

If both vectors are nonzero, their norms are nonzero, so

(11)

Therefore

(12)

Orthogonality is therefore the general algebraic form of perpendicularity.

3 Projection of one vector onto another

Suppose we want the part of v that points along a nonzero vector e.

Any vector parallel to e has the form

(13)

for some scalar c.

We therefore write

(14)

where r is the leftover or residual vector.

For ce to be the projection, the residual should contain no component along e. Therefore require

(15)

Substitute

(16)

Then

(17)

Distribute the dot product:

(18)

Move the second term to the other side:

(19)

Divide by e ⋅ e:

(20)

Thus the projection itself is

(21)

4 Special case: projection onto a unit vector

If e is a unit vector, then

(22)

Equation (20) becomes

(23)

This is why components in an orthonormal Cartesian basis are obtained by dot products.

For example,

(24)

Fourier analysis will eventually do the same thing with normalized basis functions.

5 From vectors to lists of sampled function values

Suppose we sample two functions f and g at equally spaced points

(25)

The function values form lists:

(26)

An ordinary discrete dot product is

(27)

If the points have spacing Δx, multiply by Δx:

(28)

Equation (28) is a Riemann sum. As the sample spacing becomes smaller and the number of samples increases, the sum approaches an integral:

(29)

This provides a natural bridge from the finite-dimensional vector dot product to a continuous function inner product.

6 Definition of the inner product of real functions

For real-valued functions on the interval [a,b], define

(30)

The angle brackets distinguish the function inner product from ordinary multiplication.

It is important to read the notation correctly:

(31)

is one number obtained from two entire functions.

The integration interval is part of the definition. The same pair of functions can be orthogonal on one interval and not orthogonal on another.

7 Property 1: symmetry

For real functions,

(32)

Because multiplication of real numbers is commutative,

(33)

Therefore

⟨f,g⟩	= ∫ abf(x)g(x) dx	(34)
	= ∫ abg(x)f(x) dx	(35)
	= ⟨g,f⟩.	(36)

Thus

(37)

8 Property 2: linearity

Let α and β be constants. Then

⟨αf + βh,g⟩	= ∫ ab[αf(x) + βh(x)]g(x) dx	(38)
	= α∫ abf(x)g(x) dx + β ∫ abh(x)g(x) dx	(39)
	= α⟨f,g⟩ + β⟨h,g⟩.	(40)

So the inner product is linear in each argument for real-valued functions.

9 Property 3: positivity

Take the inner product of a function with itself:

(41)

Because

(42)

we have

(43)

For the ordinary function spaces used in this series, if

(44)

then f is zero except possibly on sets that do not affect the integral. For elementary continuous functions, this simply means

(45)

throughout the interval.

10 Norm of a function

The Euclidean vector norm satisfies

(46)

The analogous function norm is

(47)

Therefore

(48)

The norm measures the overall size of the function over the interval.

It does not generally equal the maximum value of the function. It is an integrated measure of magnitude.

11 Example: norm of f(x) = x on [−1, 1]

Let

(49)

Then

∥f∥2	= ⟨f,f⟩	(50)
	= ∫ −11x2 dx.	(51)

An antiderivative is

(52)

Thus

∥f∥2	= −11	(53)
	= −	(54)
	= .	(55)

Therefore

(56)

12 Normalization of a function

A normalized function has norm one.

Given a nonzero function f, define

(57)

Then

⟨f,f⟩	=	(58)
	= ⟨f,f⟩	(59)
	= 1.	(60)

For f(x) = x on [−1, 1],

(61)

13 Orthogonal functions

Two functions are orthogonal on [a,b] when

(62)

In integral form,

(63)

The functions do not need to look geometrically perpendicular on an ordinary graph. The word orthogonal refers to their inner product.

14 Example: 1 and x on [−1, 1]

Let

(64)

Then

⟨1,x⟩	= ∫ −11xdx	(65)
	= −11	(66)
	= −	(67)
	= 0.	(68)

Therefore 1 and x are orthogonal on [−1, 1].

The cancellation occurs because x is negative on the left half and positive on the right half.

15 The interval matters

Use the same functions on [0, 1]:

⟨1,x⟩	= ∫ 01xdx	(69)
	= 01	(70)
	= .	(71)

Now the inner product is not zero.

Therefore:

• 1 and x are orthogonal on [−1, 1],
• 1 and x are not orthogonal on [0, 1].

Orthogonality belongs to the functions together with the chosen inner product and interval.

16 Even and odd symmetry already helps

Suppose f is even and g is odd on the symmetric interval [−L,L].

Then

(72)

and

(73)

Their product satisfies

[fg](−x)	= f(−x)g(−x)	(74)
	= f(x)[−g(x)]	(75)
	= −f(x)g(x).	(76)

Thus fg is odd.

The integral of an odd function over a symmetric interval is zero:

(77)

Therefore every even function is orthogonal to every odd function under this standard inner product on a symmetric interval, provided the integral exists.

This symmetry observation will save substantial work later.

17 Projection of one function onto another

Now repeat the vector argument using functions.

Suppose we want to approximate f(x) using one nonzero function ϕ(x). Write

(78)

where r is the residual.

For cϕ to contain all of the component of f along ϕ, require the residual to be orthogonal to ϕ:

(79)

From Equation (78),

(80)

Therefore

(81)

Use linearity:

(82)

Thus

(83)

Finally,

(84)

This is one of the central equations of the Fourier-series learning path.

18 If the basis function is normalized

If

(85)

then Equation (84) simplifies to

(86)

This is the function-space analogue of taking the dot product with a unit vector.

19 Worked projection example: approximate x² by a constant

Let

(87)

on [−1, 1].

Approximate it using the constant basis function

(88)

We seek

(89)

The projection coefficient is

(90)

Compute the numerator:

⟨x2, 1⟩	= ∫ −11x2 dx	(91)
	= .	(92)

Compute the denominator:

⟨1, 1⟩	= ∫ −111 dx	(93)
	= 2.	(94)

Therefore

c	=	(95)
	= .	(96)

So the projection of x² onto the constant functions is

(97)

The residual is

(98)

Check its orthogonality to the constant basis:

⟨r, 1⟩	= ∫ −11dx	(99)
	= −(2)	(100)
	= 0.	(101)

20 Why the projection constant equals the average value

For the constant basis function ϕ = 1 on [a,b],

(102)

But

(103)

Therefore

(104)

Equation (104) is exactly the average value of f over the interval.

Thus the average or DC component of a function can be understood geometrically as its projection onto the constant function.

This observation will later explain the constant term in a Fourier series.

21 Projection onto several orthogonal functions

Suppose

(105)

are mutually orthogonal:

(106)

Approximate f by

(107)

Require the residual

(108)

be orthogonal to every basis function:

(109)

Substitute the residual:

(110)

Use linearity:

(111)

Because the functions are orthogonal, every term with

(112)

vanishes. Only the n = m term remains:

(113)

Therefore

(114)

This is the general projection formula behind real Fourier coefficients.

FS03 will establish that the sine and cosine functions used in Fourier series are orthogonal on the appropriate interval. Once that is known, Equation (114) will immediately tell us how to isolate one harmonic.

22 Orthonormal functions

A collection of functions is orthonormal when

(115)

This combines two properties:

• different basis functions are orthogonal,
• each basis function has norm one.

For an orthonormal basis, Equation (114) simplifies to

(116)

This is the cleanest analogue of Cartesian vector components.

23 Pythagorean relation for orthogonal functions

For orthogonal functions f and g,

(117)

Compute the squared norm of their sum:

∥f + g∥2	= ⟨f + g,f + g⟩	(118)
	= ⟨f,f⟩ + ⟨f,g⟩ + ⟨g,f⟩ + ⟨g,g⟩.	(119)

The cross terms vanish:

(120)

Therefore

(121)

Equation (121) is the function-space Pythagorean theorem.

Later, Parseval’s theorem will be an infinite-dimensional version of this idea.

24 Angle between functions

The vector relation

(122)

suggests an analogous definition for nonzero real functions:

(123)

This provides a geometric interpretation of the inner product.

If

(124)

then

(125)

so the functions can be regarded as separated by a 90^∘ angle in function space.

The graph of one function is not literally drawn at right angles to the graph of the other. The angle belongs to the abstract geometry defined by the inner product.

25 Worked example: project x³ onto x

On [−1, 1], let

(126)

We seek

(127)

The coefficient is

(128)

The numerator is

⟨x3,x⟩	= ∫ −11x4 dx	(129)
	= −11	(130)
	= −	(131)
	= .	(132)

The denominator is

(133)

Thus

c	=	(134)
	=	(135)
	= .	(136)

Therefore the projection is

(137)

The residual is

(138)

Check orthogonality:

⟨r,x⟩	= ∫ −11xdx	(139)
	= ∫ −11dx	(140)
	= −	(141)
	= −	(142)
	= 0.	(143)

26 Weighted inner products

The ordinary inner product treats every point of the interval with the same Weight. Sometimes a problem naturally assigns greater importance to some parts of the interval than others.

Define a weighted inner product by

(144)

where the weight function satisfies

(145)

throughout the interval, apart from possible special endpoint behavior in more advanced settings.

The associated weighted norm is

(146)

Weighted inner products will become important later in generalized eigenfunction expansions and Sturm-Liouville theory.

27 Example of a weighted projection

On [0, 1], let

(147)

The weighted projection coefficient onto the constant function is

(148)

Compute the numerator:

⟨x, 1⟩w	= ∫ 01(2x)(x)(1) dx	(149)
	= 2 ∫ 01x2 dx	(150)
	= .	(151)

The denominator is

⟨1, 1⟩w	= ∫ 012xdx	(152)
	= 1.	(153)

Thus

(154)

Without the weight, the ordinary average of x on [0, 1] is

(155)

The weight w = 2x emphasizes the right side of the interval, where x is larger, so the weighted projection shifts upward to 2∕3.

28 Units of a function inner product

Suppose f and g have physical units [f] and [g], and x has units [x].

Then

(156)

Therefore an inner product is not necessarily dimensionless.

Likewise,

(157)

In mathematical treatments, variables are often nondimensionalized or the units are suppressed. In physics applications, the dimensions should still be checked.

29 Function space: what does it mean to treat a function like a vector?

A vector in ordinary three-dimensional space is specified by three components. A function requires infinitely many values to specify its complete shape.

The phrase function space refers to a set of functions treated as points or vectors in a larger abstract space.

The inner product supplies geometry to that space. It lets us define:

• length through ∥f∥,
• orthogonality through ⟨f,g⟩ = 0,
• angle through Equation (123),
• projection through Equation (84).

This geometric viewpoint is the foundation of Fourier analysis.

A rigorous treatment eventually introduces Hilbert spaces and equivalence classes of functions. That level of functional analysis is not needed yet. The elementary integral geometry is enough to derive Fourier series.

30 Why orthogonality makes coefficient extraction easy

Suppose

(158)

Take the inner product with ϕ₂:

⟨f,ϕ2⟩	= ⟨c1ϕ1 + c2ϕ2 + c3ϕ3,ϕ2⟩	(159)
	= c1⟨ϕ1,ϕ2⟩ + c2⟨ϕ2,ϕ2⟩ + c3⟨ϕ3,ϕ2⟩.	(160)

If the basis functions are mutually orthogonal,

(161)

and

(162)

Therefore

(163)

Hence

(164)

The inner product with ϕ₂ destroys every orthogonal component and leaves only the ϕ₂ component.

This is exactly what FS04 and FS05 will do with cosine and sine harmonics.

31 Preview of the Fourier connection

FS01 introduced a finite harmonic sum of the form

(165)

If the functions

(166)

are orthogonal over one full period, then each coefficient can be extracted by projection.

For example, the structure must look like

(167)

The proportionality factor depends on the norm of the cosine basis function.

We are not yet entitled to write the final Fourier coefficient formula because we have not proved the required trigonometric orthogonality identities.

That is the purpose of FS03.

32 A first three-function orthogonal example

On [−1, 1], consider

(168)

We already know

(169)

Now check

⟨1,ϕ2⟩	= ∫ −11dx	(170)
	= −	(171)
	= 0.	(172)

Next,

⟨x,ϕ2⟩	= ∫ −11xdx	(173)
	= ∫ −11dx.	(174)

The integrand is odd, so

(175)

Thus the three functions are mutually orthogonal.

This example illustrates a general strategy: new basis functions can be constructed by subtracting projections onto earlier basis functions. A systematic version of this process is the Gram-Schmidt procedure, which will be useful later but is not needed for the first Fourier derivations.

33 Self-check questions

1. What does the inner product of two functions return: another function or a scalar?
2. Why is the integration interval part of the definition of orthogonality?
3. What does it mean geometrically if

   (176)

4. Derive the projection coefficient

   (177)

   from residual orthogonality.

5. Why does the denominator disappear for a normalized basis function?
6. What is the projection of a function onto the constant basis function physically related to?
7. Why does an even function have zero inner product with an odd function over a symmetric interval?
8. Why does orthogonality make it possible to isolate one coefficient from an expansion containing many coefficients?
9. What is the difference between an orthogonal set and an orthonormal set?
10. Why are we postponing the Fourier coefficient formulas until after FS03?

34 Practice problems

1. Compute

   (178)

   on [−1, 1].

2. Determine whether x and x² are orthogonal on [−1, 1].
3. Determine whether 1 and x are orthogonal on [−2, 2] and on [0, 2].
4. Compute the norm of f(x) = 1 on [−L,L].
5. Normalize f(x) = 1 on [−L,L].
6. Compute the norm of f(x) = x on [−2, 2].
7. Project f(x) = x² onto the constant function on [0, 1].
8. Project f(x) = x⁴ onto the constant function on [−1, 1].
9. Project f(x) = x³ onto ϕ(x) = x on [−2, 2].
10. Show directly that x² − 1∕3 is orthogonal to 1 on [−1, 1].
11. Suppose ϕ₁ and ϕ₂ are orthogonal. Prove

    (179)

12. Using the weighted inner product with w(x) = x on [0, 1], project f(x) = x onto the constant function.

35 Summary

The ordinary vector dot product

(180)

has a continuous analogue for real functions:

(181)

This inner product defines the geometry needed for Fourier analysis.

Two functions are orthogonal when

(182)

The norm of a function is

(183)

The projection coefficient of f onto a nonzero basis function ϕ is

(184)

This coefficient is obtained by requiring the residual

(185)

be orthogonal to ϕ.

For an orthogonal collection {ϕ_n},

(186)

FS03 now supplies the missing ingredient: it proves in detail that distinct sine and cosine harmonics are orthogonal over a complete period.

References

References