Function Inner-Product, Orthogonality, and Projection Exercises

(Example)

FS02E1: Function Inner-Product, Orthogonality, and Projection Exercises

FS02 introduced a geometric way to think about functions. For real-valued functions on an interval [a,b], the inner product is

∫ b ⟨f,g⟩ = f (x)g(x)dx. a

(1)

From this one definition come the ideas of:

orthogonality,
function norm,
normalization,
projection,
residual orthogonality,
projection onto several mutually orthogonal functions,
weighted inner products.

This companion is designed as deliberate practice. Every problem is stated first. Hints come next. Only after that are the complete worked solutions given.

The goal is not speed. The goal is to make the projection mechanism so familiar that the Fourier coefficient derivations in FS04 and FS05 will feel like the same geometry in a trigonometric basis.

Figure 1. The exercise path. The problems begin with direct inner products and norms, then move through orthogonality and projection, and finish with weighted products and a preview of building an orthogonal basis.

1 Problems

Problem 1: a vector projection warm-up

Let

[ ] [ ] 3 2 v = 4 , e = 0 .

(2)

Find the scalar c such that

ce

(3)

is the projection of v onto e. Then compute the residual

r = v − ce

(4)

and verify

r ⋅ e = 0.

(5)

Problem 2: a basic function inner product

On [−1, 1], compute

⟨1,x2⟩.

(6)

Is the result positive, negative, or zero? What does that sign tell us geometrically?

Problem 3: even and odd functions

On [−1, 1], compute

⟨x,x2 ⟩.

(7)

Then explain why the answer could have been predicted before integrating.

Problem 4: the interval matters

For the functions

f(x ) = 1, g(x) = x,

(8)

compute the inner product on:

[−2, 2],
[0, 2].

Are the functions orthogonal on both intervals?

Problem 5: norm and normalization of a constant function

On [−L,L], where L > 0, let

f(x ) = 1.

(9)

Find:

∥f∥,
a normalized function f(x) pointing in the same function-space direction.

Verify directly that

∥ ˆf∥ = 1.

(10)

Problem 6: norm and normalization of x

On [−2, 2], compute

∥x∥.

(11)

Then construct the normalized function

ˆx(x ).

(12)

Problem 7: projection of x² onto the constant function

On [−1, 1], project

f(x) = x2

(13)

onto

ϕ(x ) = 1.

(14)

Find the projection coefficient

⟨f,ϕ⟩- c = ⟨ϕ, ϕ⟩.

(15)

Then write

f = cϕ + r

(16)

and verify explicitly that

⟨r,ϕ⟩ = 0.

(17)

Problem 8: projection onto a constant equals an average

On [0, 2], let

2 f(x) = x .

(18)

Project f onto the constant function 1. Then independently calculate the average value

1 ∫ 2 favg = ------ f(x) dx. 2 − 0 0

(19)

Compare the two results.

Problem 9: project x³ onto x

On [−2, 2], project

f(x) = x3

(20)

onto

ϕ(x) = x.

(21)

Find the coefficient c, the projected function cϕ, and the residual. Verify residual orthogonality directly.

Problem 10: rescaling a basis function

On [−1, 1], project

f(x) = x3

(22)

first onto

ϕ (x) = x, 1

(23)

and then onto

ϕ2(x) = 2x.

(24)

Compare:

the two scalar projection coefficients,
the two projected functions.

Explain why the coefficients differ even though the projected function is the same.

Problem 11: Pythagorean theorem for orthogonal functions

Suppose

⟨ϕ ,ϕ ⟩ = 0. 1 2

(25)

Prove directly that

2 2 2 2 2 ∥c1ϕ1 + c2ϕ2∥ = c1∥ϕ1∥ + c2∥ϕ2∥ .

(26)

Do not skip the cross terms. Show explicitly why they vanish.

Problem 12: angle between two functions

On [−1, 1], find the angle 𝜃 between

f(x) = 1, g(x) = x2,

(27)

using

-⟨f,-g⟩- cos𝜃 = ∥f ∥∥g ∥.

(28)

Give the exact value of cos 𝜃 and an approximate angle in degrees.

Problem 13: projection onto two orthogonal functions

On [−1, 1], use the mutually orthogonal functions

ϕ0(x) = 1, ϕ1(x) = x, ϕ2(x) = x2 − 1-. 3

(29)

Project

f(x) = x4

(30)

onto the span of these three functions.

Find the three coefficients

-⟨f,ϕk⟩- ck = ⟨ϕk,ϕk ⟩.

(31)

Then simplify the projected function.

Problem 14: verify the three-function residual

Use the projection found in Problem 13. Define

r(x) = f(x) − P (x),

(32)

where P is the projection onto the three-function span.

Verify separately that

⟨r,1⟩ = 0,

(33)

⟨r,x⟩ = 0,

(34)

and

⟨ ⟩ 2 1- r,x − 3 = 0.

(35)

Problem 15: weighted projection

Define the weighted inner product on [0, 1] by

∫ 1 ⟨f, g⟩w = xf(x)g(x) dx. 0

(36)

Project

f (x) = x

(37)

onto the constant function

ϕ(x ) = 1.

(38)

Compare the result with the unweighted projection of x onto 1 on [0, 1].

Problem 16: build a new orthogonal function by subtracting a projection

On [−1, 1], begin with

f(x) = x2

(39)

and subtract its projection onto the constant function 1.

Call the residual

q(x).

(40)

Then:

find q(x),
verify q is orthogonal to 1,
find ∥q∥,
normalize q.

This is a preview of the Gram-Schmidt idea.

2 Hints

For vector projection, use the same formula later used for functions:
$v ⋅ e c = -----. e ⋅ e$ (41)
The integrand for ⟨1,x²⟩ is simply x².
The product xx² = x³ is odd.
Orthogonality is a property of the functions together with the chosen interval.
For f = 1, the norm squared is the length of the interval.
Start from
$∫ 2 2 2 ∥x ∥ = x dx. −2$ (42)
For projection onto a constant, the denominator is the interval length.
Compare the constant-projection formula with the ordinary average-value formula.
On a symmetric interval, both numerator and denominator are even integrals.
A rescaled basis function spans the same one-dimensional subspace.
Expand the square of the norm using the inner product before using orthogonality.
Compute the three quantities ⟨f,g⟩, ∥f∥, and ∥g∥ separately.
Symmetry immediately gives the coefficient of x.
If Problem 13 is correct, the residual must be orthogonal to every basis function used in the projection.
Remember that the Weight x is part of both numerator and denominator.
Problem 7 already gives the projection of x² onto 1 on [−1, 1].

3 Worked solutions

Solution 1: vector projection warm-up

The projection coefficient is

c = v-⋅ e-. e ⋅ e

(43)

Compute the numerator:

v ⋅ e	= (3)(2) + (4)(0)	(44)
	= 6.	(45)

Compute the denominator:

e ⋅ e	= (2)(2) + (0)(0)	(46)
	= 4.	(47)

Thus

6- 3- c = 4 = 2.

(48)

The projected vector is

ce	=	(49)
	= .	(50)

The residual is

r	= v − ce	(51)
	= −	(52)
	= .	(53)

Now check orthogonality:

r ⋅ e	= (0)(2) + (4)(0)	(54)
	= 0.	(55)

This is the finite-dimensional model for every projection calculation in the rest of the companion.

Solution 2: basic function inner product

Use

∫ 1 ⟨1,x2 ⟩ = x2 dx. −1

(56)

An antiderivative is

x3- 3 .

(57)

Therefore

⟨1,x²⟩	= ₋₁¹	(58)
	= −	(59)
	= .	(60)

The inner product is positive. In the geometric language of FS02, the angle between these two nonzero functions is acute rather than 90^∘. They are not orthogonal.

Solution 3: even and odd functions

The inner product is

⟨x,x²⟩

= ∫ ₋₁¹x³ dx.

(61)

An antiderivative is

x4 ---. 4

(62)

Hence

∫ ₋₁¹x³ dx	= ₋₁¹	(63)
	= −	(64)
	= 0.	(65)

The same answer could have been predicted immediately because:

x is odd,
x² is even,
odd times even is odd,
the integral of an odd function over a symmetric interval is zero.

Thus x and x² are orthogonal on [−1, 1].

Solution 4: the interval matters

First use [−2, 2]:

⟨1,x⟩	= ∫ ₋₂²xdx	(66)
	= ₋₂²	(67)
	= 2 − 2	(68)
	= 0.	(69)

Thus 1 and x are orthogonal on [−2, 2].

Now use [0, 2]:

⟨1,x⟩	= ∫ ₀²xdx	(70)
	= ₀²	(71)
	= 2.	(72)

This is not zero. Therefore the same two formulas are not orthogonal on [0, 2].

Figure 2. Orthogonality depends on the interval. On a symmetric interval the positive and negative contributions of the odd product cancel; on [0,2] there is no matching negative part.

Solution 5: norm and normalization of a constant function

The norm is

∘ ------ ∥f ∥ = ⟨f,f ⟩.

(73)

With f = 1,

∥1∥²	= ∫ _−L^L1² dx	(74)
	= ∫ _−L^L1 dx	(75)
	= 2L.	(76)

Therefore

√ --- ∥1∥ = 2L.

(77)

Normalize by dividing by the norm:

1 fˆ(x ) = √---. 2L

(78)

Check its norm:

∥f∥²	= ∫ _−L^L²dx	(79)
	= ∫ _−L^Ldx	(80)
	= (2L)	(81)
	= 1.	(82)

Thus

∥ ˆf∥ = 1.

(83)

Figure 3. Normalization changes the length of a function-space vector without changing its direction. The normalized function has norm one.

Solution 6: norm and normalization of x

Start with

∫ 2 ∥x ∥2 = x2 dx. −2

(84)

Because x² is even,

∥x∥²	= 2 ∫ ₀²x² dx	(85)
	= 2₀²	(86)
	= 2	(87)
	= .	(88)

Hence

∥x ∥ = √4-. 3

(89)

The normalized function is

x(x)	=	(90)
	= x.	(91)

Solution 7: project x² onto the constant function

Use

⟨x2,1⟩- c = ⟨1,1⟩ .

(92)

The numerator was already found in Problem 2:

⟨x2,1⟩ = 2-. 3

(93)

The denominator is

⟨1, 1⟩	= ∫ ₋₁¹1 dx	(94)
	= 2.	(95)

Therefore

c	=	(96)
	= .	(97)

Thus the projected function is

1 P(x ) = -. 3

(98)

The residual is

r (x ) = x2 − 1. 3

(99)

Check the residual inner product with 1:

⟨r, 1⟩	= ∫ ₋₁¹dx	(100)
	= ∫ ₋₁¹x²dx − ∫ ₋₁¹1 dx	(101)
	= −(2)	(102)
	= 0.	(103)

So the residual contains no remaining constant component.

Figure 4. Projection splits x² into a constant component 1∕3 and an orthogonal residual x² − 1∕3. The residual has zero inner product with the constant basis function.

Solution 8: constant projection equals the average

Project x² onto 1 on [0, 2]:

∫ 2 2 c = -0∫-x--dx. 2 1dx 0

(104)

The numerator is

∫ ₀²x² dx	= ₀²	(105)
	= .	(106)

The denominator is

∫ 2 1 dx = 2. 0

(107)

Therefore

8∕3- 4- c = 2 = 3.

(108)

Now calculate the average directly:

f_avg	= ∫ ₀²x² dx	(109)
	=	(110)
	= .	(111)

The two results are identical. Projection onto the constant function extracts the average value on the interval.

Solution 9: project x³ onto x

Use

⟨x3,x-⟩ c = ⟨x,x⟩ .

(112)

The numerator is

⟨x³,x⟩	= ∫ ₋₂²x⁴ dx	(113)
	= 2 ∫ ₀²x⁴ dx	(114)
	= 2₀²	(115)
	= 2	(116)
	= .	(117)

The denominator is

⟨x,x⟩	= ∫ ₋₂²x² dx	(118)
	= .	(119)

Therefore

c	=	(120)
	=	(121)
	= .	(122)

The projection is

12- P (x) = 5 x.

(123)

The residual is

r(x) = x3 − 12x. 5

(124)

Now verify orthogonality:

⟨r,x⟩	= ∫ ₋₂²xdx	(125)
	= ∫ ₋₂²x⁴ dx − ∫ ₋₂²x² dx	(126)
	= −	(127)
	= −	(128)
	= 0.	(129)

Solution 10: rescaling the basis function

First project x³ onto x on [−1, 1].

The numerator is

∫ 1 x4 dx = 2. −1 5

(130)

The denominator is

∫ 1 2 2- x dx = 3. −1

(131)

Hence

c₁	=	(132)
	= .	(133)

Thus

P (x) = 3-x. 1 5

(134)

Now use ϕ₂ = 2x. The numerator is

⟨x³, 2x⟩	= 2 ∫ ₋₁¹x⁴ dx	(135)
	= .	(136)

The denominator is

⟨2x, 2x⟩	= 4 ∫ ₋₁¹x² dx	(137)
	= .	(138)

Therefore

c₂	=	(139)
	= .	(140)

But the projected function is

P₂(x)	= c₂(2x)	(141)
	= (2x)	(142)
	= x.	(143)

Thus

P = P . 1 2

(144)

The coefficient changed because the basis vector itself was rescaled. The one-dimensional subspace did not change. Projection depends on the subspace, while the numerical coefficient depends on how the basis vector is normalized.

Solution 11: Pythagorean theorem for orthogonal functions

Start from

∥c1ϕ1 + c2ϕ2∥2 = ⟨c1ϕ1 + c2ϕ2,c1ϕ1 + c2ϕ2⟩.

(145)

Use linearity in both slots for real functions:

∥c₁ϕ₁ + c₂ϕ₂∥²	= c₁²⟨ϕ₁,ϕ₁⟩	(146)
	+ c₁c₂⟨ϕ₁,ϕ₂⟩	(147)
	+ c₂c₁⟨ϕ₂,ϕ₁⟩	(148)
	+ c₂²⟨ϕ₂,ϕ₂⟩.	(149)

Orthogonality gives

⟨ϕ1,ϕ2⟩ = 0.

(150)

Symmetry gives

⟨ϕ ,ϕ ⟩ = 0. 2 1

(151)

Therefore both cross terms vanish:

∥c₁ϕ₁ + c₂ϕ₂∥²	= c₁²⟨ϕ₁,ϕ₁⟩ + c₂²⟨ϕ₂,ϕ₂⟩	(152)
	= c₁²∥ϕ₁∥² + c₂²∥ϕ₂∥².	(153)

This is exactly the function-space analogue of the ordinary Pythagorean theorem.

Solution 12: angle between 1 and x²

The numerator is

⟨1,x2⟩ = 2-. 3

(154)

The first norm is

∥1∥	=	(155)
	= .	(156)

The second norm is

∥x²∥	=	(157)
	= .	(158)

Therefore

cos 𝜃	=	(159)
	=	(160)
	= .	(161)

Thus

( √ -) −1 5 ∘ 𝜃 = cos ---- ≈ 41.81 . 3

(162)

The angle is acute because the inner product is positive.

Solution 13: projection onto three orthogonal functions

We use

2 1- ϕ0 = 1, ϕ1 = x, ϕ2 = x − 3.

(163)

Because the functions are mutually orthogonal, each coefficient can be calculated independently.

For ϕ₀:

c₀	=	(164)
	=	(165)
	= .	(166)

For ϕ₁ = x, symmetry gives

∫ 1 ⟨x4,x ⟩ = x5 dx = 0, −1

(167)

c1 = 0.

(168)

Now find c₂. The numerator is

	= ∫ ₋₁¹dx	(169)
	= −	(170)
	= −	(171)
	=	(172)
	= .	(173)

The denominator is

	= ∫ ₋₁¹²dx	(174)
	= ∫ ₋₁¹dx	(175)
	= − + (2)	(176)
	= − +	(177)
	= −	(178)
	=	(179)
	= .	(180)

Therefore

c₂	=	(181)
	=	(182)
	= .	(183)

Thus the projection is

P(x)	= +	(184)
	= + x² −	(185)
	= x² −.	(186)

Figure 5. Projection onto several mutually orthogonal functions is obtained one component at a time. Orthogonality prevents the coefficient calculations from interfering with one another.

Solution 14: verify the three-function residual

From Problem 13,

6 2 3 P (x ) = -x − ---. 7 35

(187)

Therefore

r(x)	= x⁴ − P(x)	(188)
	= x⁴ −x² + .	(189)

First check ⟨r, 1⟩:

⟨r, 1⟩	= ∫ ₋₁¹dx	(190)
	= − + (2)	(191)
	= − + .	(192)

Use denominator 35:

− +	= − +	(193)
	= 0.	(194)

Next check ⟨r,x⟩. The residual is even and x is odd, so the product is odd:

⟨r,x⟩ = 0.

(195)

Finally check ⟨r,x² − 1∕3⟩. A direct expansion gives

		(196)
	= x⁶ −x⁴ + x² −.	(197)

Therefore

	= − + −(2)	(198)
	= − + −.	(199)

Put everything over 105:

−

−

= 0.

(200)

Thus the residual is orthogonal to every function in the projection subspace.

Solution 15: weighted projection

The weighted projection coefficient is

⟨x,1⟩w c = -------. ⟨1,1⟩w

(201)

The numerator is

⟨x, 1⟩_w	= ∫ ₀¹x(x)(1) dx	(202)
	= ∫ ₀¹x² dx	(203)
	= .	(204)

The denominator is

⟨1, 1⟩_w	= ∫ ₀¹x(1)(1) dx	(205)
	= .	(206)

Thus

c_w	=	(207)
	= .	(208)

For comparison, the ordinary unweighted projection is

c	=	(209)
	= .	(210)

The weighted inner product gives greater importance to larger x, so the projected constant shifts upward from 1∕2 to 2∕3.

Figure 6. Weighting changes the geometry. With w(x) = x, the region near x = 1 contributes more strongly, so the best constant component of f(x) = x moves from 1∕2 to 2∕3.

Solution 16: subtracting a projection to make a new orthogonal function

From Problem 7, the projection of x² onto the constant function is

1 -. 3

(211)

Therefore define

q (x ) = x2 − 1. 3

(212)

By construction,

⟨q,1⟩ = 0.

(213)

We can verify this directly:

⟨q, 1⟩	= ∫ ₋₁¹dx	(214)
	= −	(215)
	= 0.	(216)

Now compute the norm squared:

∥q∥²	= ∫ ₋₁¹²dx	(217)
	= .	(218)

Thus

∘ --- ∥q∥ = 8-. 45

(219)

A normalized version is

q(x)	=	(220)
	= .	(221)

Figure 7. Subtracting the projection of x² onto the constant function produces x² − 1∕3, which is orthogonal to 1. This is the essential idea behind Gram-Schmidt orthogonalization.

4 What these exercises are preparing us to do

The repeated pattern is now clear. Given a candidate basis function ϕ, the coefficient of the component of f along ϕ is

⟨f,ϕ⟩ c = -----. ⟨ϕ, ϕ⟩

(222)

If the basis function is normalized,

⟨ϕ, ϕ⟩ = 1,

(223)

c = ⟨f,ϕ⟩.

(224)

For several mutually orthogonal functions,

∑ P (x) = c ϕ (x), k k k

(225)

with

⟨f,ϕk⟩ ck = --------. ⟨ϕk,ϕk ⟩

(226)

The next mathematical question is therefore unavoidable:

Are the sine and cosine harmonics actually orthogonal under the function inner product over a full period?

FS03 answers that question by deriving the required trigonometric integrals from first principles.

Figure 8. FS02E1 completes the projection toolkit. FS03 supplies the missing trigonometric fact: distinct sine and cosine harmonics are orthogonal over the appropriate interval.

5 Further practice

On [−3, 3], compute the norm of f(x) = 1 and normalize it.
Determine whether x and x⁴ are orthogonal on [−2, 2].
Project x⁴ onto the constant function on [−2, 2].
Project x⁵ onto x on [−1, 1].
Show that if ϕ and ψ are orthogonal, then Aϕ and Bψ are also orthogonal for any finite constants A and B.
On [−1, 1], calculate the angle between 1 and x⁴.
With the weighted inner product
$∫ 1 ⟨f,g⟩w = (1 + x )f (x)g(x)dx, 0$ (227)

project f(x) = x onto 1.
Starting from 1, x, and x², explain in words why the first two already form an orthogonal pair on [−1, 1] but x² must have a constant component removed before all three can be mutually orthogonal.

6 Summary

The function inner product

∫ b ⟨f,g⟩ = f(x)g(x) dx a

(228)

turns many familiar vector ideas into tools for functions.

A function is orthogonal to another when their inner product is zero. The norm is

∘ ------ ∥f ∥ = ⟨f,f ⟩.

(229)

Normalization gives

f fˆ= ----. ∥f∥

(230)

The projection coefficient of f onto a nonzero function ϕ is

⟨f,ϕ⟩- c = ⟨ϕ, ϕ⟩.

(231)

The residual

r = f − cϕ

(232)

satisfies

⟨r,ϕ⟩ = 0.

(233)

For mutually orthogonal basis functions, the coefficients are extracted independently.

A weighted inner product changes the geometry by changing how strongly different parts of the interval contribute.

FS03 now specializes these general ideas to sine and cosine functions and proves their orthogonality one integral at a time.

References

[1] PhysicsLibrary, FS02: Inner Products of Functions.

[2] E. Kreyszig, Advanced Engineering Mathematics, Wiley.

[3] R. Haberman, Applied Partial Differential Equations with Fourier Series and Boundary Value Problems, Pearson.

"Function Inner-Product, Orthogonality, and Projection Exercises" is owned by bloftin.
(view preamble)

FS02E1: Function Inner-Product, Orthogonality, and Projection Exercises

1 Problems

Problem 1: a vector projection warm-up

Problem 2: a basic function inner product

Problem 3: even and odd functions

Problem 4: the interval matters

Problem 5: norm and normalization of a constant function

Problem 6: norm and normalization of x

Problem 7: projection of x2 onto the constant function

Problem 8: projection onto a constant equals an average

Problem 9: project x3 onto x

Problem 10: rescaling a basis function

Problem 11: Pythagorean theorem for orthogonal functions

Problem 12: angle between two functions

Problem 13: projection onto two orthogonal functions

Problem 14: verify the three-function residual

Problem 15: weighted projection

Problem 16: build a new orthogonal function by subtracting a projection

2 Hints

3 Worked solutions

Solution 1: vector projection warm-up

Solution 2: basic function inner product

Solution 3: even and odd functions

Solution 4: the interval matters

Solution 5: norm and normalization of a constant function

Solution 6: norm and normalization of x

Solution 7: project x2 onto the constant function

Solution 8: constant projection equals the average

Solution 9: project x3 onto x

Solution 10: rescaling the basis function

Solution 11: Pythagorean theorem for orthogonal functions

Solution 12: angle between 1 and x2

Solution 13: projection onto three orthogonal functions

Solution 14: verify the three-function residual

Solution 15: weighted projection

Solution 16: subtracting a projection to make a new orthogonal function

4 What these exercises are preparing us to do

5 Further practice

6 Summary

References

References

Problem 7: projection of x² onto the constant function

Problem 9: project x³ onto x

Solution 7: project x² onto the constant function

Solution 9: project x³ onto x

Solution 12: angle between 1 and x²