1
00:00:10,510 --> 00:00:11,010
OK.

2
00:00:11,010 --> 00:00:11,635
Shall we start?

3
00:00:11,635 --> 00:00:15,360
This is the second
lecture on eigenvalues.

4
00:00:15,360 --> 00:00:18,350
So the first lecture was --

5
00:00:18,350 --> 00:00:22,350
reached the key equation,
A x equal lambda x.

6
00:00:22,350 --> 00:00:27,090
x is the eigenvector and
lambda's the eigenvalue.

7
00:00:27,090 --> 00:00:30,220
Now to use that.

8
00:00:30,220 --> 00:00:36,330
And the, the good way
to, after we've found --

9
00:00:36,330 --> 00:00:39,400
so, so job one is to
find the eigenvalues

10
00:00:39,400 --> 00:00:41,990
and find the eigenvectors.

11
00:00:41,990 --> 00:00:44,750
Now after we've found them,
what do we do with them?

12
00:00:44,750 --> 00:00:50,360
Well, the good way to see that
is diagonalize the matrix.

13
00:00:50,360 --> 00:00:52,850
So the matrix is A.

14
00:00:52,850 --> 00:00:55,130
And I want to show
-- first of all,

15
00:00:55,130 --> 00:00:57,190
this is like the basic fact.

16
00:00:57,190 --> 00:00:58,720
This, this formula.

17
00:00:58,720 --> 00:01:03,210
That's, that's the key
to today's lecture.

18
00:01:03,210 --> 00:01:06,810
This matrix A, I
put its eigenvectors

19
00:01:06,810 --> 00:01:09,660
in the columns of a matrix S.

20
00:01:09,660 --> 00:01:12,740
So S will be the
eigenvector matrix.

21
00:01:12,740 --> 00:01:19,560
And I want to look at this
magic combination S inverse A S.

22
00:01:19,560 --> 00:01:22,830
So can I show you how that --

23
00:01:22,830 --> 00:01:24,200
what happens there?

24
00:01:24,200 --> 00:01:27,900
And notice, there's
an S inverse.

25
00:01:27,900 --> 00:01:33,690
We have to be able to invert
this eigenvector matrix S.

26
00:01:33,690 --> 00:01:38,330
So for that, we need n
independent eigenvectors.

27
00:01:38,330 --> 00:01:41,440
So that's the, that's the case.

28
00:01:41,440 --> 00:01:42,130
OK.

29
00:01:42,130 --> 00:01:53,740
So suppose we have n linearly
independent eigenvectors

30
00:01:53,740 --> 00:02:11,830
of A. Put them in the
columns of this matrix S.

31
00:02:11,830 --> 00:02:15,430
So I'm naturally going to call
that the eigenvector matrix,

32
00:02:15,430 --> 00:02:18,730
because it's got the
eigenvectors in its columns.

33
00:02:18,730 --> 00:02:21,730
And all I want to do is
show you what happens

34
00:02:21,730 --> 00:02:24,960
when you multiply A times S.

35
00:02:24,960 --> 00:02:29,200
So A times S.

36
00:02:29,200 --> 00:02:34,490
So this is A times the matrix
with the first eigenvector

37
00:02:34,490 --> 00:02:37,300
in its first column,
the second eigenvector

38
00:02:37,300 --> 00:02:42,510
in its second column, the n-th
eigenvector in its n-th column.

39
00:02:42,510 --> 00:02:46,300
And how I going to do this
matrix multiplication?

40
00:02:46,300 --> 00:02:49,890
Well, certainly I'll do
it a column at a time.

41
00:02:49,890 --> 00:02:52,510
And what do I get.

42
00:02:52,510 --> 00:02:55,480
A times the first column
gives me the first column

43
00:02:55,480 --> 00:02:57,820
of the answer, but what is it?

44
00:02:57,820 --> 00:03:00,010
That's an eigenvector.

45
00:03:00,010 --> 00:03:04,280
A times x1 is equal to
the lambda times the x1.

46
00:03:04,280 --> 00:03:06,870
And that lambda's we're
-- we'll call lambda one,

47
00:03:06,870 --> 00:03:09,590
of course.

48
00:03:09,590 --> 00:03:11,420
So that's the first column.

49
00:03:11,420 --> 00:03:14,000
Ax1 is the same
as lambda one x1.

50
00:03:14,000 --> 00:03:16,730
A x2 is lambda two x2.

51
00:03:16,730 --> 00:03:22,120
So on, along to in the n-th
column we now how lambda n xn.

52
00:03:24,690 --> 00:03:27,110
Looking good, but
the next step is even

53
00:03:27,110 --> 00:03:28,120
better.

54
00:03:28,120 --> 00:03:31,280
So for the next step,
I want to separate out

55
00:03:31,280 --> 00:03:36,100
those eigenvalues, those,
those multiplying numbers,

56
00:03:36,100 --> 00:03:38,880
from the x-s.

57
00:03:38,880 --> 00:03:41,920
So then I'll have
just what I want.

58
00:03:41,920 --> 00:03:42,530
OK.

59
00:03:42,530 --> 00:03:44,990
So how, how I going
to separate out?

60
00:03:44,990 --> 00:03:47,150
So that, that
number lambda one is

61
00:03:47,150 --> 00:03:49,760
multiplying the first column.

62
00:03:49,760 --> 00:03:52,840
So if I want to factor it
out of the first column,

63
00:03:52,840 --> 00:03:55,400
I better put --

64
00:03:55,400 --> 00:03:58,830
here is going to
be x1, and that's

65
00:03:58,830 --> 00:04:02,180
going to multiply
this matrix lambda

66
00:04:02,180 --> 00:04:05,050
one in the first
entry and all zeros.

67
00:04:05,050 --> 00:04:08,230
Do you see that that,
that's going to come out

68
00:04:08,230 --> 00:04:10,220
right for the first column?

69
00:04:10,220 --> 00:04:13,370
Because w- we remember how --
how we're going back to that

70
00:04:13,370 --> 00:04:15,310
original punchline.

71
00:04:15,310 --> 00:04:21,260
That if I want a number
to multiply x1 then

72
00:04:21,260 --> 00:04:24,310
I can do it by putting
x1 in that column,

73
00:04:24,310 --> 00:04:27,280
in the first column, and
putting that number there.

74
00:04:27,280 --> 00:04:28,850
Th- u- what I
going to have here?

75
00:04:28,850 --> 00:04:30,040
I'm going to have lambda --

76
00:04:30,040 --> 00:04:33,800
I'm going to have x1, x2, ...

77
00:04:33,800 --> 00:04:34,890
,xn.

78
00:04:34,890 --> 00:04:37,150
These are going to
be my columns again.

79
00:04:37,150 --> 00:04:39,740
I'm getting S back again.

80
00:04:39,740 --> 00:04:41,380
I'm getting S back again.

81
00:04:41,380 --> 00:04:44,560
But now what's it multiplied
by, on the right it's

82
00:04:44,560 --> 00:04:48,080
multiplied by?

83
00:04:48,080 --> 00:04:54,730
If I want lambda n xn in the
last column, how do I do it?

84
00:04:54,730 --> 00:04:57,670
Well, the last column
here will be --

85
00:04:57,670 --> 00:05:02,630
I'll take the last column,
use these coefficients,

86
00:05:02,630 --> 00:05:05,050
put the lambda n
down there, and it

87
00:05:05,050 --> 00:05:09,970
will multiply that n-th column
and give me lambda n xn.

88
00:05:09,970 --> 00:05:13,130
There, there you see matrix
multiplication just working

89
00:05:13,130 --> 00:05:14,850
for us.

90
00:05:14,850 --> 00:05:17,410
So I started with A S.

91
00:05:17,410 --> 00:05:21,010
I wrote down what it meant,
A times each eigenvector.

92
00:05:21,010 --> 00:05:23,690
That gave me lambda
time the eigenvector.

93
00:05:23,690 --> 00:05:25,700
And then when I peeled
off the lambdas,

94
00:05:25,700 --> 00:05:30,690
they were on the right-hand
side, so I've got S, my matrix,

95
00:05:30,690 --> 00:05:31,750
back again.

96
00:05:31,750 --> 00:05:45,500
And this matrix, this diagonal
matrix, the eigenvalue matrix,

97
00:05:45,500 --> 00:05:49,210
and I call it capital lambda.

98
00:05:49,210 --> 00:05:52,830
Using capital letters
for matrices and lambda

99
00:05:52,830 --> 00:05:54,980
to prompt me that
it's, that it's

100
00:05:54,980 --> 00:05:56,850
eigenvalues that are in there.

101
00:05:56,850 --> 00:05:58,840
So you see that the
eigenvalues are just

102
00:05:58,840 --> 00:06:01,480
sitting down that diagonal?

103
00:06:01,480 --> 00:06:07,340
If I had a column x2 here,
I would want the lambda two

104
00:06:07,340 --> 00:06:10,970
in the two two position,
in the diagonal position,

105
00:06:10,970 --> 00:06:15,140
to multiply that x2 and
give me the lambda two x2.

106
00:06:15,140 --> 00:06:16,730
That's my formula.

107
00:06:16,730 --> 00:06:19,770
A S is S lambda.

108
00:06:23,000 --> 00:06:23,720
OK.

109
00:06:23,720 --> 00:06:27,530
That's the -- you see,
it's just a calculation.

110
00:06:27,530 --> 00:06:31,390
Now -- I mentioned, and
I have to mention again,

111
00:06:31,390 --> 00:06:37,210
this business about n
independent eigenvectors.

112
00:06:37,210 --> 00:06:40,050
As it stands, this is
all fine, whether --

113
00:06:40,050 --> 00:06:43,240
I mean, I could be repeating
the same eigenvector, but --

114
00:06:43,240 --> 00:06:45,550
I'm not interested in that.

115
00:06:45,550 --> 00:06:50,600
I want to be able to invert S,
and that's where this comes in.

116
00:06:50,600 --> 00:06:53,690
This n independent
eigenvectors business

117
00:06:53,690 --> 00:06:56,990
comes in to tell me that
that matrix is invertible.

118
00:06:56,990 --> 00:07:01,380
So let me, on the next board,
write down what I've got.

119
00:07:01,380 --> 00:07:06,780
A S equals S lambda.

120
00:07:06,780 --> 00:07:11,890
And now I'm, I can multiply
on the left by S inverse.

121
00:07:11,890 --> 00:07:13,839
So this is really --

122
00:07:17,970 --> 00:07:20,660
I can do that, provided
S is invertible.

123
00:07:20,660 --> 00:07:26,320
Provided my assumption of n
independent eigenvectors is

124
00:07:26,320 --> 00:07:27,260
satisfied.

125
00:07:27,260 --> 00:07:31,470
And I mentioned at the end of
last time, and I'll say again,

126
00:07:31,470 --> 00:07:35,930
that there's a small
number of matrices for --

127
00:07:35,930 --> 00:07:38,680
that don't have n
independent eigenvectors.

128
00:07:38,680 --> 00:07:42,580
So I've got to discuss
that, that technical point.

129
00:07:42,580 --> 00:07:48,030
But the great -- the most
matrices that we see have n di-

130
00:07:48,030 --> 00:07:51,050
n independent eigenvectors,
and we can diagonalize.

131
00:07:51,050 --> 00:07:54,090
This is diagonalization.

132
00:07:54,090 --> 00:07:57,890
I could also write
it, and I often will,

133
00:07:57,890 --> 00:07:59,680
the other way round.

134
00:07:59,680 --> 00:08:03,140
If I multiply on the
right by S inverse,

135
00:08:03,140 --> 00:08:06,130
if I took this equation at the
top and multiplied on the right

136
00:08:06,130 --> 00:08:08,090
by S inverse, I could --

137
00:08:08,090 --> 00:08:10,900
I would have A left here.

138
00:08:10,900 --> 00:08:15,790
Now S inverse is
coming from the right.

139
00:08:15,790 --> 00:08:18,530
So can you keep
those two straight?

140
00:08:18,530 --> 00:08:21,491
A multiplies its eigenvectors,
that's how I keep them

141
00:08:21,491 --> 00:08:21,990
straight.

142
00:08:21,990 --> 00:08:24,530
So A multiplies S.

143
00:08:24,530 --> 00:08:26,020
A multiplies S.

144
00:08:26,020 --> 00:08:30,700
And then this S inverse makes
the whole thing diagonal.

145
00:08:30,700 --> 00:08:33,750
And this is another way
of saying the same thing,

146
00:08:33,750 --> 00:08:36,700
putting the Ss on the
other side of the equation.

147
00:08:36,700 --> 00:08:40,179
A is S lambda S inverse.

148
00:08:40,179 --> 00:08:44,520
So that's the, that's
the new factorization.

149
00:08:44,520 --> 00:08:50,550
That's the replacement for L U
from elimination or Q R for --

150
00:08:50,550 --> 00:08:52,210
from Gram-Schmidt.

151
00:08:52,210 --> 00:08:56,850
And notice that the matrix --
so it's, it's a matrix times

152
00:08:56,850 --> 00:09:00,420
a diagonal matrix times the
inverse of the first one.

153
00:09:00,420 --> 00:09:02,410
It's, that's the
combination that we'll

154
00:09:02,410 --> 00:09:05,530
see throughout this chapter.

155
00:09:05,530 --> 00:09:09,650
This combination with
an S and an S inverse.

156
00:09:09,650 --> 00:09:10,250
OK.

157
00:09:10,250 --> 00:09:14,080
Can I just begin to use that?

158
00:09:14,080 --> 00:09:17,730
For example, what
about A squared?

159
00:09:17,730 --> 00:09:20,880
What are the eigenvalues and
eigenvectors of A squared?

160
00:09:20,880 --> 00:09:23,040
That's a straightforward
question with a,

161
00:09:23,040 --> 00:09:25,380
with an absolutely clean answer.

162
00:09:25,380 --> 00:09:28,930
So let me, let me
consider A squared.

163
00:09:28,930 --> 00:09:33,560
So I start with A
x equal lambda x.

164
00:09:33,560 --> 00:09:36,070
And I'm headed for A squared.

165
00:09:36,070 --> 00:09:38,610
So let me multiply
both sides by A.

166
00:09:38,610 --> 00:09:41,290
That's one way to get
A squared on the left.

167
00:09:41,290 --> 00:09:46,190
So -- I should write
these if-s in here.

168
00:09:46,190 --> 00:09:51,010
If A x equals lambda x,
then I multiply by A,

169
00:09:51,010 --> 00:09:56,200
so I get A squared x equals
-- well, I'm multiplying by A,

170
00:09:56,200 --> 00:09:58,670
so that's lambda A x.

171
00:09:58,670 --> 00:10:02,520
That lambda was a number, so
I just put it on the left.

172
00:10:02,520 --> 00:10:06,740
And what do I -- tell me how
to make that look better.

173
00:10:06,740 --> 00:10:10,220
What have I got
here for if, if A

174
00:10:10,220 --> 00:10:13,220
has the eigenvalue
lambda and eigenvector

175
00:10:13,220 --> 00:10:16,010
x, what's up with A squared?

176
00:10:16,010 --> 00:10:18,430
A squared x, I just
multiplied by A,

177
00:10:18,430 --> 00:10:23,070
but now for Ax I'm going
to substitute lambda x.

178
00:10:23,070 --> 00:10:25,010
So I've got lambda squared x.

179
00:10:28,120 --> 00:10:31,370
So from that simple
calculation, I --

180
00:10:31,370 --> 00:10:37,330
my conclusion is that the
eigenvalues of A squared are

181
00:10:37,330 --> 00:10:39,310
lambda squared.

182
00:10:39,310 --> 00:10:40,384
And the eigenvectors --

183
00:10:40,384 --> 00:10:41,550
I always think about both of

184
00:10:41,550 --> 00:10:42,150
those.

185
00:10:42,150 --> 00:10:44,390
What can I say about
the eigenvalues?

186
00:10:44,390 --> 00:10:45,560
They're squared.

187
00:10:45,560 --> 00:10:48,220
What can I say about
the eigenvectors?

188
00:10:48,220 --> 00:10:49,730
They're the same.

189
00:10:49,730 --> 00:10:54,090
The same x as in -- as for A.

190
00:10:54,090 --> 00:11:00,320
Now let me see that
also from this formula.

191
00:11:00,320 --> 00:11:05,160
How can I see what A squared is
looking like from this formula?

192
00:11:05,160 --> 00:11:08,480
So let me -- that
was one way to do it.

193
00:11:08,480 --> 00:11:12,510
Let me do it by just
taking A squared from that.

194
00:11:12,510 --> 00:11:18,990
A squared is S lambda S
inverse -- that's A --

195
00:11:18,990 --> 00:11:22,620
times S lambda S inverse
-- that's A, which is?

196
00:11:25,530 --> 00:11:30,170
This is the beauty of
eigenvalues, eigenvectors.

197
00:11:30,170 --> 00:11:34,000
Having that S inverse
and S is the identity,

198
00:11:34,000 --> 00:11:40,220
so I've got S lambda
squared S inverse.

199
00:11:40,220 --> 00:11:43,380
Do you see what
that's telling me?

200
00:11:43,380 --> 00:11:46,570
It's, it's telling me the same
thing that I just learned here,

201
00:11:46,570 --> 00:11:49,630
but in the -- in a matrix form.

202
00:11:49,630 --> 00:11:52,720
It's telling me that
the S is the same,

203
00:11:52,720 --> 00:11:57,130
the eigenvectors are the
same, but the eigenvalues

204
00:11:57,130 --> 00:11:58,530
are squared.

205
00:11:58,530 --> 00:12:01,020
Because this is --
what's lambda squared?

206
00:12:01,020 --> 00:12:02,800
That's still diagonal.

207
00:12:02,800 --> 00:12:05,180
It's got little
lambda one squared,

208
00:12:05,180 --> 00:12:07,190
lambda two squared,
down to lambda n

209
00:12:07,190 --> 00:12:09,280
squared o- on that diagonal.

210
00:12:09,280 --> 00:12:13,760
Those are the eigenvalues, as
we just learned, of A squared.

211
00:12:13,760 --> 00:12:14,500
OK.

212
00:12:14,500 --> 00:12:19,590
So -- somehow those eigenvalues
and eigenvectors are really

213
00:12:19,590 --> 00:12:25,060
giving you a way to --

214
00:12:25,060 --> 00:12:29,460
see what's going
on inside a matrix.

215
00:12:29,460 --> 00:12:32,100
Of course I can
continue that for --

216
00:12:32,100 --> 00:12:36,270
to the K-th power,
A to the K-th power.

217
00:12:36,270 --> 00:12:39,370
If I multiply, if I have
K of these together,

218
00:12:39,370 --> 00:12:42,920
do you see how S inverse
S will keep canceling

219
00:12:42,920 --> 00:12:44,900
in the, in the inside?

220
00:12:44,900 --> 00:12:48,740
I'll have the S outside
at the far left,

221
00:12:48,740 --> 00:12:54,390
and lambda will be in there
K times, and S inverse.

222
00:12:54,390 --> 00:12:56,670
So what's that telling me?

223
00:12:56,670 --> 00:12:59,420
That's telling me
that the eigenvalues

224
00:12:59,420 --> 00:13:04,340
of A, of A to the K-th
power are the K-th powers.

225
00:13:04,340 --> 00:13:08,390
The eigenvalues of A cubed are
the cubes of the eigenvalues of

226
00:13:08,390 --> 00:13:15,000
A. And the eigenvectors
are the same, the same.

227
00:13:15,000 --> 00:13:15,600
OK.

228
00:13:15,600 --> 00:13:20,750
In other words, eigenvalues
and eigenvectors

229
00:13:20,750 --> 00:13:25,910
give a great way to understand
the powers of a matrix.

230
00:13:25,910 --> 00:13:28,380
If I take the
square of a matrix,

231
00:13:28,380 --> 00:13:30,540
or the hundredth
power of a matrix,

232
00:13:30,540 --> 00:13:34,780
the pivots are all
over the place.

233
00:13:34,780 --> 00:13:39,690
L U, if I multiply L U times
L U times L U times L U

234
00:13:39,690 --> 00:13:44,610
a hundred times, I've
got a hundred L Us.

235
00:13:44,610 --> 00:13:46,370
I can't do anything with them.

236
00:13:46,370 --> 00:13:50,180
But when I multiply S
lambda S inverse by itself,

237
00:13:50,180 --> 00:13:54,720
when I look at the eigenvector
picture a hundred times,

238
00:13:54,720 --> 00:13:59,810
I get a hundred or ninety-nine
of these guys canceling out

239
00:13:59,810 --> 00:14:03,390
inside, and I get
A to the hundredth

240
00:14:03,390 --> 00:14:05,900
is S lambda to the
hundredth S inverse.

241
00:14:05,900 --> 00:14:09,870
I mean, eigenvalues
tell you about powers

242
00:14:09,870 --> 00:14:16,290
of a matrix in a way that we had
no way to approach previously.

243
00:14:16,290 --> 00:14:21,220
For example, when does --

244
00:14:21,220 --> 00:14:24,790
when do the powers of
a matrix go to zero?

245
00:14:24,790 --> 00:14:29,330
I would call that
matrix stable, maybe.

246
00:14:29,330 --> 00:14:31,940
So I could write down a theorem.

247
00:14:31,940 --> 00:14:36,720
I'll write it as a theorem
just to use that word

248
00:14:36,720 --> 00:14:40,350
to emphasize that here I'm
getting this great fact

249
00:14:40,350 --> 00:14:42,610
from this eigenvalue picture.

250
00:14:42,610 --> 00:14:43,230
OK.

251
00:14:43,230 --> 00:14:53,720
A to the K approaches zero as K
goes, as K gets bigger if what?

252
00:14:53,720 --> 00:14:57,840
What's the w- how can
I tell, for a matrix A,

253
00:14:57,840 --> 00:14:59,850
if its powers go to zero?

254
00:15:02,650 --> 00:15:07,520
What's -- somewhere inside that
matrix is that information.

255
00:15:07,520 --> 00:15:11,510
That information is not
present in the pivots.

256
00:15:11,510 --> 00:15:13,270
It's present in the eigenvalues.

257
00:15:13,270 --> 00:15:17,010
What do I need for the -- to
know that if I take higher

258
00:15:17,010 --> 00:15:20,170
and higher powers of A, that
this matrix gets smaller

259
00:15:20,170 --> 00:15:21,030
and smaller?

260
00:15:21,030 --> 00:15:24,330
Well, S and S inverse
are not moving.

261
00:15:24,330 --> 00:15:26,810
So it's this guy that
has to get small.

262
00:15:26,810 --> 00:15:29,720
And that's easy to
-- to understand.

263
00:15:29,720 --> 00:15:32,595
The requirement is
all eigenvalues --

264
00:15:35,330 --> 00:15:38,040
so what is the requirement?

265
00:15:38,040 --> 00:15:41,170
The eigenvalues have
to be less than one.

266
00:15:41,170 --> 00:15:45,310
Now I have to wrote
that absolute value,

267
00:15:45,310 --> 00:15:48,360
because those eigenvalues
could be negative,

268
00:15:48,360 --> 00:15:50,550
they could be complex numbers.

269
00:15:50,550 --> 00:15:53,170
So I'm taking the
absolute value.

270
00:15:53,170 --> 00:15:56,750
If all of those are below one.

271
00:15:56,750 --> 00:16:05,000
That's, in fact, we
practically see why.

272
00:16:05,000 --> 00:16:13,100
And let me just say that I'm
operating on one assumption

273
00:16:13,100 --> 00:16:15,740
here, and I got to
keep remembering

274
00:16:15,740 --> 00:16:18,690
that that assumption
is still present.

275
00:16:18,690 --> 00:16:21,420
That assumption was that
I had a full set of,

276
00:16:21,420 --> 00:16:24,370
of n independent eigenvectors.

277
00:16:24,370 --> 00:16:30,830
If I don't have that, then
this approach is not working.

278
00:16:30,830 --> 00:16:37,090
So again, a pure eigenvalue
approach, eigenvector approach,

279
00:16:37,090 --> 00:16:40,470
needs n independent
eigenvectors.

280
00:16:40,470 --> 00:16:42,900
If we don't have n
independent eigenvectors,

281
00:16:42,900 --> 00:16:46,490
we can't diagonalize the matrix.

282
00:16:46,490 --> 00:16:50,820
We can't get to a
diagonal matrix.

283
00:16:50,820 --> 00:16:55,960
This diagonalization
is only possible

284
00:16:55,960 --> 00:16:58,581
if S inverse makes sense.

285
00:16:58,581 --> 00:16:59,080
OK.

286
00:16:59,080 --> 00:17:02,800
Can I, can I follow
up on that point now?

287
00:17:02,800 --> 00:17:07,099
So you see why -- what we
get and, and why we want it,

288
00:17:07,099 --> 00:17:11,490
because we get information about
the powers of a matrix just

289
00:17:11,490 --> 00:17:14,940
immediately from
the eigenvalues.

290
00:17:14,940 --> 00:17:15,500
OK.

291
00:17:15,500 --> 00:17:22,329
Now let me follow up on this,
business of which matrices

292
00:17:22,329 --> 00:17:25,030
are diagonalizable.

293
00:17:25,030 --> 00:17:28,220
Sorry about that long word.

294
00:17:28,220 --> 00:17:32,940
So a matrix is, is sure -- so
here's, here's the main point.

295
00:17:32,940 --> 00:17:37,600
A is sure to be --

296
00:17:37,600 --> 00:17:50,110
to have N independent
eigenvectors and, and be --

297
00:17:50,110 --> 00:18:00,210
now here comes that word
-- diagonalizable if, if --

298
00:18:00,210 --> 00:18:05,810
so we might as well get the
nice case out in the open.

299
00:18:05,810 --> 00:18:13,330
The nice case is when -- if
all the lambdas are different.

300
00:18:19,520 --> 00:18:28,326
That means, that means
no repeated eigenvalues.

301
00:18:30,920 --> 00:18:32,000
OK.

302
00:18:32,000 --> 00:18:34,410
That's the nice case.

303
00:18:34,410 --> 00:18:39,970
If my matrix, and most -- if
I do a random matrix in Matlab

304
00:18:39,970 --> 00:18:43,140
and compute its eigenvalues --

305
00:18:43,140 --> 00:18:55,540
so if I computed if I took
eig of rand of ten ten, gave,

306
00:18:55,540 --> 00:18:58,640
gave that Matlab command, the --

307
00:18:58,640 --> 00:19:01,190
we'd get a random
ten by ten matrix,

308
00:19:01,190 --> 00:19:03,990
we would get a list of
its ten eigenvalues,

309
00:19:03,990 --> 00:19:08,040
and they would be different.

310
00:19:08,040 --> 00:19:10,470
They would be distinct
is the best word.

311
00:19:10,470 --> 00:19:13,880
I would have -- a random matrix
will have ten distinct --

312
00:19:13,880 --> 00:19:18,090
a ten by ten matrix will have
ten distinct eigenvalues.

313
00:19:18,090 --> 00:19:25,500
And if it does, the eigenvectors
are automatically independent.

314
00:19:25,500 --> 00:19:26,930
So that's a nice fact.

315
00:19:26,930 --> 00:19:29,730
I'll refer you to the
text for the proof.

316
00:19:29,730 --> 00:19:36,360
That, that A is sure to have
n independent eigenvectors

317
00:19:36,360 --> 00:19:41,610
if the eigenvalues
are different, if.

318
00:19:41,610 --> 00:19:43,890
If all the, if all
eigenvalues are different.

319
00:19:43,890 --> 00:19:47,810
It's just if some
lambdas are repeated,

320
00:19:47,810 --> 00:19:50,560
then I have to
look more closely.

321
00:19:50,560 --> 00:19:55,050
If an eigenvalue is repeated, I
have to look, I have to count,

322
00:19:55,050 --> 00:19:56,260
I have to check.

323
00:19:56,260 --> 00:19:59,560
Has it got -- say it's
repeated three times.

324
00:19:59,560 --> 00:20:02,030
So what's a
possibility for the --

325
00:20:02,030 --> 00:20:05,893
so here is the, here is
the repeated possibility.

326
00:20:11,000 --> 00:20:16,490
And, and let me
emphasize the conclusion.

327
00:20:16,490 --> 00:20:21,410
That if I have repeated
eigenvalues, I may or may not,

328
00:20:21,410 --> 00:20:35,271
I may or may not have, have
n independent eigenvectors.

329
00:20:35,271 --> 00:20:35,770
I might.

330
00:20:35,770 --> 00:20:41,240
I, I, you know, this isn't
a completely negative case.

331
00:20:41,240 --> 00:20:43,260
The identity matrix --

332
00:20:43,260 --> 00:20:46,380
suppose I take the ten
by ten identity matrix.

333
00:20:46,380 --> 00:20:50,750
What are the eigenvalues
of that matrix?

334
00:20:50,750 --> 00:20:55,370
So just, just take the
easiest matrix, the identity.

335
00:20:55,370 --> 00:21:00,470
If I look for its
eigenvalues, they're all ones.

336
00:21:00,470 --> 00:21:04,410
So that eigenvalue one
is repeated ten times.

337
00:21:04,410 --> 00:21:07,340
But there's no shortage of
eigenvectors for the identity

338
00:21:07,340 --> 00:21:08,250
matrix.

339
00:21:08,250 --> 00:21:10,760
In fact, every vector
is an eigenvector.

340
00:21:10,760 --> 00:21:13,610
So I can take ten
independent vectors.

341
00:21:13,610 --> 00:21:16,530
Oh, well, what happens
to everything --

342
00:21:16,530 --> 00:21:18,590
if A is the identity
matrix, let's

343
00:21:18,590 --> 00:21:21,650
just think that one
through in our head.

344
00:21:21,650 --> 00:21:27,440
If A is the identity
matrix, then it's

345
00:21:27,440 --> 00:21:28,830
got plenty of eigenvectors.

346
00:21:28,830 --> 00:21:30,910
I choose ten
independent vectors.

347
00:21:30,910 --> 00:21:32,560
They're the columns of S.

348
00:21:32,560 --> 00:21:37,380
And, and what do I get
from S inverse A S?

349
00:21:37,380 --> 00:21:39,400
I get I again, right?

350
00:21:39,400 --> 00:21:42,210
If A is the identity -- and
of course that's the correct

351
00:21:42,210 --> 00:21:43,790
lambda.

352
00:21:43,790 --> 00:21:46,790
The matrix was already diagonal.

353
00:21:46,790 --> 00:21:48,970
So if the matrix is
already diagonal,

354
00:21:48,970 --> 00:21:53,800
then the, the lambda is
the same as the matrix.

355
00:21:53,800 --> 00:21:56,380
A diagonal matrix has
got its eigenvalues

356
00:21:56,380 --> 00:21:59,170
sitting right there
in front of you.

357
00:21:59,170 --> 00:22:01,790
Now if it's triangular,
the eigenvalues

358
00:22:01,790 --> 00:22:04,820
are still sitting
there, but so let's

359
00:22:04,820 --> 00:22:08,460
take a case where
it's triangular.

360
00:22:08,460 --> 00:22:14,870
Suppose A is like,
two one two zero.

361
00:22:17,920 --> 00:22:23,290
So there's a case that's
going to be trouble.

362
00:22:23,290 --> 00:22:25,040
There's a case that's
going to be trouble.

363
00:22:25,040 --> 00:22:26,360
First of all, what are the --

364
00:22:26,360 --> 00:22:29,410
I mean, we just --

365
00:22:29,410 --> 00:22:31,190
if we start with a
matrix, the first thing

366
00:22:31,190 --> 00:22:32,890
we do, practically
without thinking

367
00:22:32,890 --> 00:22:36,130
is compute the eigenvalues
and eigenvectors.

368
00:22:36,130 --> 00:22:36,630
OK.

369
00:22:36,630 --> 00:22:38,360
So what are the eigenvalues?

370
00:22:38,360 --> 00:22:41,130
You can tell me right
away what they are.

371
00:22:41,130 --> 00:22:43,880
They're two and two, right.

372
00:22:43,880 --> 00:22:47,770
It's a triangular matrix, so
when I do this determinant,

373
00:22:47,770 --> 00:22:51,680
shall I do this determinant
of A minus lambda I?

374
00:22:51,680 --> 00:22:59,990
I'll get this two minus lambda
one zero two minus lambda,

375
00:22:59,990 --> 00:23:01,700
right?

376
00:23:01,700 --> 00:23:06,890
I take that determinant, so I
make those into vertical bars

377
00:23:06,890 --> 00:23:09,130
to mean determinant.

378
00:23:09,130 --> 00:23:10,810
And what's the determinant?

379
00:23:10,810 --> 00:23:13,140
It's two minus lambda squared.

380
00:23:13,140 --> 00:23:14,570
What are the roots?

381
00:23:14,570 --> 00:23:17,410
Lambda equal two twice.

382
00:23:17,410 --> 00:23:22,640
So the eigenvalues are
lambda equals two and two.

383
00:23:22,640 --> 00:23:23,580
OK, fine.

384
00:23:23,580 --> 00:23:26,810
Now the next step,
find the eigenvectors.

385
00:23:26,810 --> 00:23:31,940
So I look for eigenvectors, and
what do I find for this guy?

386
00:23:31,940 --> 00:23:33,530
Eigenvectors for
this guy, when I

387
00:23:33,530 --> 00:23:38,340
subtract two minus the
identity, so A minus two

388
00:23:38,340 --> 00:23:42,280
I has zeros here.

389
00:23:45,420 --> 00:23:48,740
And I'm looking
for the null space.

390
00:23:48,740 --> 00:23:50,390
What's, what are
the eigenvectors?

391
00:23:50,390 --> 00:23:56,540
They're the -- the null
space of A minus lambda I.

392
00:23:56,540 --> 00:23:59,200
The null space is
only one dimensional.

393
00:23:59,200 --> 00:24:03,700
This is a case where I don't
have enough eigenvectors.

394
00:24:03,700 --> 00:24:07,940
My algebraic
multiplicity is two.

395
00:24:07,940 --> 00:24:10,520
I would say, when
I see, when I count

396
00:24:10,520 --> 00:24:16,210
how often the
eigenvalue is repeated,

397
00:24:16,210 --> 00:24:18,410
that's the algebraic
multiplicity.

398
00:24:18,410 --> 00:24:20,650
That's the multiplicity,
how many times

399
00:24:20,650 --> 00:24:22,770
is it the root of
the polynomial?

400
00:24:22,770 --> 00:24:28,340
My polynomial is two
minus lambda squared.

401
00:24:28,340 --> 00:24:30,040
It's a double root.

402
00:24:30,040 --> 00:24:33,240
So my algebraic
multiplicity is two.

403
00:24:33,240 --> 00:24:37,110
But the geometric multiplicity,
which looks for vectors,

404
00:24:37,110 --> 00:24:42,130
looks for eigenvectors, and
-- which means the null space

405
00:24:42,130 --> 00:24:46,500
of this thing, and the
only eigenvector is one

406
00:24:46,500 --> 00:24:47,360
zero.

407
00:24:47,360 --> 00:24:50,140
That's in the null space.

408
00:24:50,140 --> 00:24:52,600
Zero one is not
in the null space.

409
00:24:52,600 --> 00:24:54,540
The null space is
only one dimensional.

410
00:24:54,540 --> 00:24:58,960
So there's a matrix, my --
this A or the original A,

411
00:24:58,960 --> 00:25:02,310
that are not diagonalizable.

412
00:25:02,310 --> 00:25:06,360
I can't find two
independent eigenvectors.

413
00:25:06,360 --> 00:25:08,090
There's only one.

414
00:25:08,090 --> 00:25:08,700
OK.

415
00:25:08,700 --> 00:25:11,710
So that's the case that I'm --

416
00:25:11,710 --> 00:25:15,520
that's a case that I'm
not really handling.

417
00:25:15,520 --> 00:25:19,590
For example, when I
wrote down up here

418
00:25:19,590 --> 00:25:24,490
that the powers went to zero if
the eigenvalues were below one,

419
00:25:24,490 --> 00:25:29,210
I didn't really handle that
case of repeated eigenvalues,

420
00:25:29,210 --> 00:25:33,980
because my reasoning was
based on this formula.

421
00:25:33,980 --> 00:25:36,420
And this formula is based on
n independent eigenvectors.

422
00:25:36,420 --> 00:25:36,920
OK.

423
00:25:36,920 --> 00:25:45,610
Just to say then, there are
some matrices that we're, that,

424
00:25:45,610 --> 00:25:48,730
that we don't cover
through diagonalization,

425
00:25:48,730 --> 00:25:51,070
but the great majority we do.

426
00:25:51,070 --> 00:25:51,720
OK.

427
00:25:51,720 --> 00:25:54,040
And we, we're
always OK if we have

428
00:25:54,040 --> 00:25:56,550
different distinct eigenvalues.

429
00:25:56,550 --> 00:26:02,390
OK, that's the, like,
the typical case.

430
00:26:02,390 --> 00:26:04,660
Because for each
eigenvalue there's

431
00:26:04,660 --> 00:26:07,140
at least one eigenvector.

432
00:26:07,140 --> 00:26:11,530
The algebraic multiplicity here
is one for every eigenvalue

433
00:26:11,530 --> 00:26:14,080
and the geometric
multiplicity is one.

434
00:26:14,080 --> 00:26:15,580
There's one eigenvector.

435
00:26:15,580 --> 00:26:17,650
And they are independent.

436
00:26:17,650 --> 00:26:18,150
OK.

437
00:26:18,150 --> 00:26:18,650
OK.

438
00:26:21,690 --> 00:26:26,390
Now let me come back to
the important case, when,

439
00:26:26,390 --> 00:26:27,770
when we're OK.

440
00:26:27,770 --> 00:26:31,820
The important case, when
we are diagonalizable.

441
00:26:31,820 --> 00:26:38,060
Let me, look at --

442
00:26:38,060 --> 00:26:42,455
so -- let me solve
this equation.

443
00:26:46,680 --> 00:26:49,460
The equation will be each --

444
00:26:49,460 --> 00:26:57,146
I start with some -- start
with a given vector u0.

445
00:27:02,390 --> 00:27:06,080
And then my equation
is at every step,

446
00:27:06,080 --> 00:27:11,600
I multiply what I have by A.

447
00:27:11,600 --> 00:27:16,550
That, that equation ought
to be simple to handle.

448
00:27:19,940 --> 00:27:21,940
And I'd like to be
able to solve it.

449
00:27:21,940 --> 00:27:26,840
How would I find -- if I start
with a vector u0 and I multiply

450
00:27:26,840 --> 00:27:31,470
by A a hundred times,
what have I got?

451
00:27:31,470 --> 00:27:35,310
Well, I could certainly write
down a formula for the answer,

452
00:27:35,310 --> 00:27:39,295
so what, what -- so u1 is A u0.

453
00:27:42,170 --> 00:27:45,800
And u2 is -- what's u2 then?

454
00:27:45,800 --> 00:27:52,350
u2, I multiply -- u2 I get from
u1 by another multiplying by A,

455
00:27:52,350 --> 00:27:55,830
so I've got A twice.

456
00:27:55,830 --> 00:28:02,120
And my formula is
uk, after k steps,

457
00:28:02,120 --> 00:28:07,580
I've multiplied by A k
times the original u0.

458
00:28:07,580 --> 00:28:11,220
You see what I'm doing?

459
00:28:11,220 --> 00:28:14,050
The next section is
going to solve systems

460
00:28:14,050 --> 00:28:17,550
of differential equations.

461
00:28:17,550 --> 00:28:19,690
I'm going to have derivatives.

462
00:28:19,690 --> 00:28:23,370
This section is the nice one.

463
00:28:23,370 --> 00:28:26,190
It solves difference equations.

464
00:28:26,190 --> 00:28:28,550
I would call that a
difference equation.

465
00:28:28,550 --> 00:28:33,550
It's -- at first order, I would
call that a first-order system,

466
00:28:33,550 --> 00:28:40,150
because it connects only --
it only goes up one level.

467
00:28:40,150 --> 00:28:43,180
And I -- it's a system
because these are vectors

468
00:28:43,180 --> 00:28:45,960
and that's a matrix.

469
00:28:45,960 --> 00:28:48,470
And the solution is just that.

470
00:28:48,470 --> 00:28:49,090
OK.

471
00:28:49,090 --> 00:28:55,160
But, that's a nice formula.

472
00:28:55,160 --> 00:28:57,500
That's the, like, the
most compact formula

473
00:28:57,500 --> 00:29:01,630
I could ever get. u100 would
be A to the one hundred u0.

474
00:29:01,630 --> 00:29:06,480
But how would I
actually find u100?

475
00:29:06,480 --> 00:29:11,520
How would I find -- how would
I discover what u100 is?

476
00:29:11,520 --> 00:29:13,760
Let me, let me show you how.

477
00:29:16,620 --> 00:29:18,630
Here's the idea.

478
00:29:18,630 --> 00:29:23,090
If -- so to solve, to
really solve -- shall I say,

479
00:29:23,090 --> 00:29:26,920
to really solve --

480
00:29:26,920 --> 00:29:34,500
to really solve it, I would
take this initial vector u0

481
00:29:34,500 --> 00:29:39,420
and I would write it as a
combination of eigenvectors.

482
00:29:39,420 --> 00:29:47,320
To really solve, write u
nought as a combination,

483
00:29:47,320 --> 00:29:50,660
say certain amount of
the first eigenvector

484
00:29:50,660 --> 00:29:53,480
plus a certain amount of
the second eigenvector

485
00:29:53,480 --> 00:29:55,820
plus a certain amount
of the last eigenvector.

486
00:30:01,790 --> 00:30:04,740
Now multiply by A.

487
00:30:04,740 --> 00:30:07,350
You want to -- you got to
see the magic of eigenvectors

488
00:30:07,350 --> 00:30:08,520
working here.

489
00:30:08,520 --> 00:30:10,360
Multiply by A.

490
00:30:10,360 --> 00:30:13,910
So Au0 is what?

491
00:30:13,910 --> 00:30:16,930
So A times that.

492
00:30:16,930 --> 00:30:18,800
A times -- so what's A --

493
00:30:18,800 --> 00:30:21,390
I can separate it out
into n separate pieces,

494
00:30:21,390 --> 00:30:23,430
and that's the whole point.

495
00:30:23,430 --> 00:30:28,800
That each of those pieces is
going in its own merry way.

496
00:30:28,800 --> 00:30:31,320
Each of those pieces
is an eigenvector,

497
00:30:31,320 --> 00:30:35,810
and when I multiply by A,
what does this piece become?

498
00:30:35,810 --> 00:30:38,450
So that's some amount
of the first --

499
00:30:38,450 --> 00:30:41,030
let's suppose the eigenvectors
are normalized to be unit

500
00:30:41,030 --> 00:30:41,530
vectors.

501
00:30:44,750 --> 00:30:48,530
So that says what
the eigenvector is.

502
00:30:48,530 --> 00:30:51,340
It's a --

503
00:30:51,340 --> 00:30:55,220
And I need some multiple
of it to produce u0.

504
00:30:55,220 --> 00:30:56,120
OK.

505
00:30:56,120 --> 00:30:59,470
Now when I multiply
by A, what do I get?

506
00:30:59,470 --> 00:31:04,350
I get c1, which is just
a factor, times Ax1,

507
00:31:04,350 --> 00:31:07,865
but Ax1 is lambda one x1.

508
00:31:10,780 --> 00:31:17,060
When I multiply this by
A, I get c2 lambda two x2.

509
00:31:17,060 --> 00:31:20,740
And here I get cn lambda n xn.

510
00:31:20,740 --> 00:31:27,980
And suppose I multiply by A
to the hundredth power now.

511
00:31:27,980 --> 00:31:30,840
Can we, having done it,
multiplied by A, let's

512
00:31:30,840 --> 00:31:32,890
multiply by A to the hundredth.

513
00:31:32,890 --> 00:31:36,380
What happens to this first term
when I multiply by A to the one

514
00:31:36,380 --> 00:31:38,130
hundredth?

515
00:31:38,130 --> 00:31:41,620
It's got that factor
lambda to the hundredth.

516
00:31:41,620 --> 00:31:42,890
That's the key.

517
00:31:42,890 --> 00:31:48,440
That -- that's what I mean
by going its own merry way.

518
00:31:48,440 --> 00:31:52,320
It, it is pure eigenvector.

519
00:31:52,320 --> 00:31:55,850
It's exactly in a direction
where multiplication by A

520
00:31:55,850 --> 00:31:59,200
just brings in a scalar
factor, lambda one.

521
00:31:59,200 --> 00:32:02,240
So a hundred times brings
in this a hundred times.

522
00:32:02,240 --> 00:32:06,080
Hundred times lambda two,
hundred times lambda n.

523
00:32:06,080 --> 00:32:08,830
Actually, we're -- what
are we seeing here?

524
00:32:08,830 --> 00:32:15,040
We're seeing, this
same, lambda capital

525
00:32:15,040 --> 00:32:19,570
lambda to the hundredth as in
the, as in the diagonalization.

526
00:32:19,570 --> 00:32:22,350
And we're seeing
the S matrix, the,

527
00:32:22,350 --> 00:32:24,730
the matrix S of eigenvectors.

528
00:32:24,730 --> 00:32:29,440
That's what this has got to
-- this has got to amount to.

529
00:32:29,440 --> 00:32:40,030
A lambda to the hundredth power
times an S times this vector c

530
00:32:40,030 --> 00:32:43,490
that's telling us
how much of each one

531
00:32:43,490 --> 00:32:45,010
is in the original thing.

532
00:32:45,010 --> 00:32:49,010
So if, if I had to really
find the hundredth power,

533
00:32:49,010 --> 00:32:54,200
I would take u0, I would
expand it as a combination

534
00:32:54,200 --> 00:32:57,210
of eigenvectors --
this is really S,

535
00:32:57,210 --> 00:33:01,680
the eigenvector matrix, times
c, the, the coefficient vector.

536
00:33:04,240 --> 00:33:07,310
And then I would
immediately then,

537
00:33:07,310 --> 00:33:10,950
by inserting these hundredth
powers of eigenvalues,

538
00:33:10,950 --> 00:33:15,490
I'd have the answer.

539
00:33:15,490 --> 00:33:17,880
So -- huh, there must be --

540
00:33:17,880 --> 00:33:20,570
oh, let's see, OK.

541
00:33:20,570 --> 00:33:22,970
It's -- so, yeah.

542
00:33:22,970 --> 00:33:30,790
So if u100 is A to the hundredth
times u0, and u0 is S c --

543
00:33:30,790 --> 00:33:36,160
then you see this formula
is just this formula,

544
00:33:36,160 --> 00:33:40,840
which is the way I would
actually get hold of this,

545
00:33:40,840 --> 00:33:44,690
of this u100, which is --

546
00:33:44,690 --> 00:33:47,180
let me put it here.

547
00:33:47,180 --> 00:33:48,030
u100.

548
00:33:48,030 --> 00:33:51,070
The way I would actually
get hold of that, see what,

549
00:33:51,070 --> 00:33:57,400
what the solution is after
a hundred steps, would be --

550
00:33:57,400 --> 00:34:05,960
expand the initial vector
into eigenvectors and let each

551
00:34:05,960 --> 00:34:10,020
eigenvector go its own way,
multiplying by a hundred at --

552
00:34:10,020 --> 00:34:13,400
by lambda at every step,
and therefore by lambda

553
00:34:13,400 --> 00:34:16,030
to the hundredth power
after a hundred steps.

554
00:34:16,030 --> 00:34:18,050
Can I do an example?

555
00:34:18,050 --> 00:34:20,260
So that's the formulas.

556
00:34:20,260 --> 00:34:22,540
Now let me take an example.

557
00:34:22,540 --> 00:34:29,090
I'll use the Fibonacci
sequence as an example.

558
00:34:29,090 --> 00:34:31,590
So, so Fibonacci example.

559
00:34:39,830 --> 00:34:43,050
You remember the
Fibonacci numbers?

560
00:34:43,050 --> 00:34:48,150
If we start with one
and one as F0 -- oh,

561
00:34:48,150 --> 00:34:50,280
I think I start
with zero, maybe.

562
00:34:50,280 --> 00:34:54,550
Let zero and one
be the first ones.

563
00:34:54,550 --> 00:34:58,550
So there's F0 and F1, the
first two Fibonacci numbers.

564
00:34:58,550 --> 00:35:02,840
Then what's the rule
for Fibonacci numbers?

565
00:35:02,840 --> 00:35:04,130
Ah, they're the sum.

566
00:35:04,130 --> 00:35:08,030
The next one is the sum
of those, so it's one.

567
00:35:08,030 --> 00:35:11,110
The next one is the sum
of those, so it's two.

568
00:35:11,110 --> 00:35:14,010
The next one is the sum
of those, so it's three.

569
00:35:14,010 --> 00:35:16,190
Well, it looks like one
two three four five,

570
00:35:16,190 --> 00:35:19,350
but somehow it's not
going to do that way.

571
00:35:19,350 --> 00:35:21,380
The next one is five, right.

572
00:35:21,380 --> 00:35:22,640
Two and three makes five.

573
00:35:22,640 --> 00:35:26,090
The next one is eight.

574
00:35:26,090 --> 00:35:28,370
The next one is thirteen.

575
00:35:28,370 --> 00:35:33,245
And the one hundredth
Fibonacci number is what?

576
00:35:35,920 --> 00:35:37,600
That's my question.

577
00:35:37,600 --> 00:35:40,680
How could I get a formula
for the hundredth number?

578
00:35:40,680 --> 00:35:44,470
And, for example, how could
I answer the question,

579
00:35:44,470 --> 00:35:47,740
how fast are they growing?

580
00:35:47,740 --> 00:35:52,650
How fast are those
Fibonacci numbers growing?

581
00:35:52,650 --> 00:35:54,070
They're certainly growing.

582
00:35:54,070 --> 00:35:56,270
It's not a stable case.

583
00:35:56,270 --> 00:35:59,030
Whatever the eigenvalues
of whatever matrix it is,

584
00:35:59,030 --> 00:36:00,720
they're not smaller than one.

585
00:36:00,720 --> 00:36:02,540
These numbers are growing.

586
00:36:02,540 --> 00:36:04,450
But how fast are they growing?

587
00:36:04,450 --> 00:36:10,070
The answer lies
in the eigenvalue.

588
00:36:10,070 --> 00:36:12,450
So I've got to find the
matrix, so let me write down

589
00:36:12,450 --> 00:36:14,495
the Fibonacci rule.

590
00:36:17,610 --> 00:36:22,245
F(k+2) = F(k+1)+F k, right?

591
00:36:25,210 --> 00:36:28,280
Now that's not in my --

592
00:36:28,280 --> 00:36:32,420
I want to write that
as uk plus one and Auk.

593
00:36:32,420 --> 00:36:38,920
But right now what I've got is
a single equation, not a system,

594
00:36:38,920 --> 00:36:41,140
and it's second-order.

595
00:36:41,140 --> 00:36:44,290
It's like having a second-order
differential equation

596
00:36:44,290 --> 00:36:45,810
with second derivatives.

597
00:36:45,810 --> 00:36:47,580
I want to get first derivatives.

598
00:36:47,580 --> 00:36:49,200
Here I want to get
first differences.

599
00:36:49,200 --> 00:36:55,910
So the way, the way to do it
is to introduce uk will be

600
00:36:55,910 --> 00:36:57,960
a vector --

601
00:36:57,960 --> 00:36:59,125
see, a small trick.

602
00:37:01,920 --> 00:37:05,330
Let uk be a vector,
F(k+1) and Fk.

603
00:37:08,230 --> 00:37:12,680
So I'm going to get a two
by two system, first order,

604
00:37:12,680 --> 00:37:16,890
instead of a one -- instead of
a scalar system, second order,

605
00:37:16,890 --> 00:37:18,300
by a simple trick.

606
00:37:18,300 --> 00:37:22,820
I'm just going to add in an
equation F(k+1) equals F(k+1).

607
00:37:22,820 --> 00:37:28,980
That will be my second equation.

608
00:37:28,980 --> 00:37:33,940
Then this is my system,
this is my unknown,

609
00:37:33,940 --> 00:37:38,690
and what's my one step equation?

610
00:37:38,690 --> 00:37:45,120
So, so now u(k+1), that's --
so u(k+1) is the left side,

611
00:37:45,120 --> 00:37:47,620
and what have I got
here on the right side?

612
00:37:47,620 --> 00:37:52,530
I've got some matrix
multiplying uk.

613
00:37:52,530 --> 00:37:56,510
Can you, do -- can you
see that all right?

614
00:37:56,510 --> 00:37:59,450
if you can see it, then you
can tell me what the matrix is.

615
00:37:59,450 --> 00:38:02,860
Do you see that I'm
taking my system here.

616
00:38:02,860 --> 00:38:06,550
I artificially made
it into a system.

617
00:38:06,550 --> 00:38:10,540
I artificially made the
unknown into a vector.

618
00:38:10,540 --> 00:38:14,260
And now I'm ready to look
at and see what the matrix

619
00:38:14,260 --> 00:38:15,020
is.

620
00:38:15,020 --> 00:38:20,240
So do you see the left side,
u(k+1) is F(k+2) F(k+1),

621
00:38:20,240 --> 00:38:21,940
that's just what I want.

622
00:38:21,940 --> 00:38:25,590
On the right side, this
remember, this uk here --

623
00:38:25,590 --> 00:38:29,960
let me for the moment
put it as F(k+1) Fk.

624
00:38:29,960 --> 00:38:33,080
So what's the matrix?

625
00:38:33,080 --> 00:38:41,380
Well, that has a one and a one,
and that has a one and a zero.

626
00:38:41,380 --> 00:38:43,080
There's the matrix.

627
00:38:43,080 --> 00:38:47,880
Do you see that that gives
me the right-hand side?

628
00:38:47,880 --> 00:38:52,360
So there's the matrix A.

629
00:38:52,360 --> 00:38:56,810
And this is our friend uk.

630
00:38:56,810 --> 00:39:00,650
So we've got -- so
that simple trick --

631
00:39:00,650 --> 00:39:03,900
changed the second-order
scalar problem

632
00:39:03,900 --> 00:39:05,730
to a first-order system.

633
00:39:05,730 --> 00:39:08,750
Two b- u- with two unknowns.

634
00:39:08,750 --> 00:39:10,040
With a matrix.

635
00:39:10,040 --> 00:39:13,100
And now what do I do?

636
00:39:13,100 --> 00:39:16,240
Well, before I even think,
I find its eigenvalues

637
00:39:16,240 --> 00:39:18,170
and eigenvectors.

638
00:39:18,170 --> 00:39:21,170
So what are the eigenvalues and
eigenvectors of that matrix?

639
00:39:23,820 --> 00:39:24,320
Let's see.

640
00:39:24,320 --> 00:39:27,083
I always -- first let me just,
like, think for a minute.

641
00:39:29,720 --> 00:39:35,440
It's two by two, so this
shouldn't be impossible to do.

642
00:39:35,440 --> 00:39:37,020
Let's do it.

643
00:39:37,020 --> 00:39:37,670
OK.

644
00:39:37,670 --> 00:39:43,170
So my matrix, again,
is one one one zero.

645
00:39:46,170 --> 00:39:49,070
It's symmetric, by the way.

646
00:39:49,070 --> 00:39:56,070
So what I will eventually
know about symmetric matrices

647
00:39:56,070 --> 00:39:59,140
is that the eigenvalues
will come out real.

648
00:39:59,140 --> 00:40:02,290
I won't get any
complex numbers here.

649
00:40:02,290 --> 00:40:06,210
And the eigenvectors,
once I get those,

650
00:40:06,210 --> 00:40:08,520
actually will be orthogonal.

651
00:40:08,520 --> 00:40:11,190
But two by two, I'm
more interested in what

652
00:40:11,190 --> 00:40:13,740
the actual numbers are.

653
00:40:13,740 --> 00:40:16,230
What do I know about
the two numbers?

654
00:40:16,230 --> 00:40:18,190
Well, should do
you want me to find

655
00:40:18,190 --> 00:40:19,820
this determinant of A minus

656
00:40:19,820 --> 00:40:20,629
lambda I?

657
00:40:20,629 --> 00:40:21,128
Sure.

658
00:40:23,880 --> 00:40:27,910
So it's the determinant of
one minus lambda one one zero,

659
00:40:27,910 --> 00:40:28,410
right?

660
00:40:31,900 --> 00:40:33,400
Minus lambda, yes.

661
00:40:33,400 --> 00:40:33,994
God.

662
00:40:33,994 --> 00:40:34,493
OK.

663
00:40:38,030 --> 00:40:40,110
OK.

664
00:40:40,110 --> 00:40:42,240
There'll be two eigenvalues.

665
00:40:42,240 --> 00:40:45,160
What will -- tell me again
what I know about the two

666
00:40:45,160 --> 00:40:47,550
eigenvalues before
I go any further.

667
00:40:47,550 --> 00:40:49,590
Tell me something about
these two eigenvalues.

668
00:40:49,590 --> 00:40:51,770
What do they add up to?

669
00:40:51,770 --> 00:40:55,860
Lambda one plus lambda two is?

670
00:40:55,860 --> 00:41:02,390
Is the same as the trace down
the diagonal of the matrix.

671
00:41:02,390 --> 00:41:04,660
One and zero is one.

672
00:41:04,660 --> 00:41:08,320
So lambda one plus lambda two
should come out to be one.

673
00:41:08,320 --> 00:41:10,710
And lambda one times
lambda one times lambda two

674
00:41:10,710 --> 00:41:13,300
should come out to be
the determinant, which

675
00:41:13,300 --> 00:41:15,360
is minus one.

676
00:41:15,360 --> 00:41:18,440
So I'm expecting the
eigenvalues to add to one

677
00:41:18,440 --> 00:41:20,570
and to multiply to minus one.

678
00:41:20,570 --> 00:41:22,720
But let's just see
it happen here.

679
00:41:22,720 --> 00:41:26,680
If I multiply this out, I get --
that times that'll be a lambda

680
00:41:26,680 --> 00:41:30,290
squared minus lambda minus one.

681
00:41:30,290 --> 00:41:30,790
Good.

682
00:41:33,830 --> 00:41:36,570
Lambda squared minus
lambda minus one.

683
00:41:36,570 --> 00:41:43,250
Actually, I -- you see the b-
compare that with the original

684
00:41:43,250 --> 00:41:48,655
equation that I started with.

685
00:41:48,655 --> 00:41:49,780
F(k+2) - F(k+1)-Fk is zero.

686
00:41:49,780 --> 00:42:00,330
The recursion that -- that the
Fibonacci numbers satisfy is

687
00:42:00,330 --> 00:42:05,140
somehow showing up directly here
for the eigenvalues when we set

688
00:42:05,140 --> 00:42:06,230
that to zero.

689
00:42:06,230 --> 00:42:06,730
WK.

690
00:42:06,730 --> 00:42:09,530
Let's solve.

691
00:42:09,530 --> 00:42:14,200
Well, I would like to be able
to factor that, that quadratic,

692
00:42:14,200 --> 00:42:17,450
but I'm better off to use
the quadratic formula.

693
00:42:17,450 --> 00:42:19,880
Lambda is -- let's see.

694
00:42:19,880 --> 00:42:25,860
Minus b is one plus or minus
the square root of b squared,

695
00:42:25,860 --> 00:42:30,650
which is one, minus four
times that times that,

696
00:42:30,650 --> 00:42:33,740
which is plus four, over two.

697
00:42:37,614 --> 00:42:39,030
So that's the
square root of five.

698
00:42:42,600 --> 00:42:50,230
So the eigenvalues are
lambda one is one half of one

699
00:42:50,230 --> 00:42:57,000
plus square root of five, and
lambda two is one half of one

700
00:42:57,000 --> 00:42:59,220
minus square root of five.

701
00:42:59,220 --> 00:43:04,630
And sure enough, they -- those
add up to one and they multiply

702
00:43:04,630 --> 00:43:06,890
to give minus one.

703
00:43:06,890 --> 00:43:07,390
OK.

704
00:43:07,390 --> 00:43:09,400
Those are the two eigenvalues.

705
00:43:09,400 --> 00:43:12,250
How -- what are those
numbers approximately?

706
00:43:12,250 --> 00:43:18,060
Square root of five,
well, it's more than two

707
00:43:18,060 --> 00:43:19,030
but less than three.

708
00:43:19,030 --> 00:43:19,860
Hmm.

709
00:43:19,860 --> 00:43:25,330
It'd be nice to
know these numbers.

710
00:43:25,330 --> 00:43:30,480
I think, I think that -- so that
number comes out bigger than

711
00:43:30,480 --> 00:43:30,980
one, right?

712
00:43:30,980 --> 00:43:31,640
That's right.

713
00:43:31,640 --> 00:43:35,070
This number comes
out bigger than one.

714
00:43:35,070 --> 00:43:38,210
It's about one point six
one eight or something.

715
00:43:42,610 --> 00:43:44,700
Not exactly, but.

716
00:43:44,700 --> 00:43:48,300
And suppose it's one point six.

717
00:43:48,300 --> 00:43:52,390
Just, like, I think so.

718
00:43:52,390 --> 00:43:54,800
Then what's lambda two?

719
00:43:54,800 --> 00:43:57,870
Is, is lambda two
positive or negative?

720
00:43:57,870 --> 00:44:01,430
Negative, right, because I'm
-- it's, obviously negative,

721
00:44:01,430 --> 00:44:07,420
and I knew that the
-- so it's minus --

722
00:44:07,420 --> 00:44:16,720
and they add up to one, so minus
point six one eight, I guess.

723
00:44:16,720 --> 00:44:17,220
OK.

724
00:44:17,220 --> 00:44:17,800
A- and some more.

725
00:44:17,800 --> 00:44:18,520
Those are the two eigenvalues.

726
00:44:18,520 --> 00:44:19,830
One eigenvalue bigger than one,
one eigenvalue smaller than

727
00:44:19,830 --> 00:44:20,330
one.

728
00:44:20,330 --> 00:44:22,480
Actually, that's a great
situation to be in.

729
00:44:22,480 --> 00:44:25,430
Of course, the
eigenvalues are different,

730
00:44:25,430 --> 00:44:29,340
so there's no doubt whatever --
is this matrix diagonalizable?

731
00:44:32,270 --> 00:44:35,280
Is this matrix diagonalizable,
that original matrix A?

732
00:44:35,280 --> 00:44:35,990
Sure.

733
00:44:35,990 --> 00:44:38,030
We've got two
distinct eigenvalues

734
00:44:38,030 --> 00:44:44,294
and we can find the
eigenvectors in a moment.

735
00:44:44,294 --> 00:44:46,460
But they'll be independent,
we'll be diagonalizable.

736
00:44:46,460 --> 00:44:54,790
And now, you, you can already
answer my very first question.

737
00:44:54,790 --> 00:44:59,530
How fast are those Fibonacci
numbers increasing?

738
00:44:59,530 --> 00:45:01,080
How -- those --
they're increasing,

739
00:45:01,080 --> 00:45:01,810
right?

740
00:45:01,810 --> 00:45:03,970
They're not doubling
at every step.

741
00:45:03,970 --> 00:45:07,330
Let me -- let's look
again at these numbers.

742
00:45:07,330 --> 00:45:09,580
Five, eight, thirteen,
it's not obvious.

743
00:45:09,580 --> 00:45:14,060
The next one would be
twenty-one, thirty-four.

744
00:45:14,060 --> 00:45:20,924
So to get some idea of
what F one hundred is,

745
00:45:20,924 --> 00:45:21,840
can you give me any --

746
00:45:21,840 --> 00:45:24,820
I mean the crucial number --

747
00:45:24,820 --> 00:45:32,280
so it -- these --
it's approximately --

748
00:45:32,280 --> 00:45:37,970
what's controlling the growth
of these Fibonacci numbers?

749
00:45:37,970 --> 00:45:39,630
It's the eigenvalues.

750
00:45:39,630 --> 00:45:43,031
And which eigenvalue is
controlling that growth?

751
00:45:43,031 --> 00:45:43,530
The big one.

752
00:45:43,530 --> 00:45:50,380
So F100 will be approximately
some constant, c1 I guess,

753
00:45:50,380 --> 00:45:56,110
times this lambda one, this
one plus square root of five

754
00:45:56,110 --> 00:46:01,300
over two, to the
hundredth power.

755
00:46:01,300 --> 00:46:04,560
And the two hundredth F -- in
other words, the eigenvalue --

756
00:46:04,560 --> 00:46:08,950
the Fibonacci numbers are
growing by about that factor.

757
00:46:08,950 --> 00:46:13,780
Do you see that we, we've got
precise information about the,

758
00:46:13,780 --> 00:46:18,230
about the Fibonacci numbers
out of the eigenvalues?

759
00:46:18,230 --> 00:46:18,940
OK.

760
00:46:18,940 --> 00:46:21,880
And again, why is that true?

761
00:46:21,880 --> 00:46:26,750
Let me go over to this board
and s- show what I'm doing here.

762
00:46:26,750 --> 00:46:30,720
The -- the original initial
value is some combination

763
00:46:30,720 --> 00:46:31,730
of eigenvectors.

764
00:46:35,520 --> 00:46:39,470
And then when we start -- when
we start going out the theories

765
00:46:39,470 --> 00:46:42,580
of Fibonacci numbers, when
we start multiplying by A

766
00:46:42,580 --> 00:46:45,980
a hundred times, it's this
lambda one to the hundredth.

767
00:46:45,980 --> 00:46:51,070
This term is, is the
one that's taking over.

768
00:46:51,070 --> 00:46:54,920
It's -- I mean, that's big, like
one point six to the hundredth

769
00:46:54,920 --> 00:46:55,880
power.

770
00:46:55,880 --> 00:47:00,610
The second term is
practically nothing, right?

771
00:47:00,610 --> 00:47:04,300
The point six, or minus point
six, to the hundredth power

772
00:47:04,300 --> 00:47:08,180
is an extremely small,
extremely small number.

773
00:47:08,180 --> 00:47:11,410
So this is -- there're
only two terms,

774
00:47:11,410 --> 00:47:13,020
because we're two by two.

775
00:47:13,020 --> 00:47:16,430
This number is -- this
piece of it is there,

776
00:47:16,430 --> 00:47:21,270
but it's, it's disappearing,
where this piece is there

777
00:47:21,270 --> 00:47:23,890
and it's growing and
controlling everything.

778
00:47:23,890 --> 00:47:27,230
So, so really the --
we're doing, like,

779
00:47:27,230 --> 00:47:29,100
problems that are evolving.

780
00:47:29,100 --> 00:47:33,390
We're doing dynamic
u- instead of Ax=b,

781
00:47:33,390 --> 00:47:35,440
that's a static problem.

782
00:47:35,440 --> 00:47:36,930
We're now we're doing dynamics.

783
00:47:36,930 --> 00:47:39,740
A, A squared, A cubed,
things are evolving in

784
00:47:39,740 --> 00:47:40,440
time.

785
00:47:40,440 --> 00:47:44,660
And the eigenvalues are
the crucial, numbers.

786
00:47:44,660 --> 00:47:45,640
OK.

787
00:47:45,640 --> 00:47:52,490
I guess to complete
this, I better

788
00:47:52,490 --> 00:47:56,420
write down the eigenvectors.

789
00:47:56,420 --> 00:47:59,160
So we should complete
the, the whole process

790
00:47:59,160 --> 00:48:01,200
by finding the eigenvectors.

791
00:48:01,200 --> 00:48:03,820
OK, well, I have to --
up in the corner, then,

792
00:48:03,820 --> 00:48:07,670
I have to look at
A minus lambda I.

793
00:48:07,670 --> 00:48:15,800
So A minus lambda I is this one
minus lambda one one and minus

794
00:48:15,800 --> 00:48:16,930
lambda.

795
00:48:16,930 --> 00:48:19,990
And now can we spot an
eigenvector out of that?

796
00:48:19,990 --> 00:48:23,070
That's, that's, for
these two lambdas,

797
00:48:23,070 --> 00:48:24,415
this matrix is singular.

798
00:48:27,380 --> 00:48:30,350
I guess the eigenvector -- two
by two ought to be, I mean,

799
00:48:30,350 --> 00:48:31,260
easy.

800
00:48:31,260 --> 00:48:33,960
So if I know that this
matrix is singular,

801
00:48:33,960 --> 00:48:37,100
then u- seems to
me the eigenvector

802
00:48:37,100 --> 00:48:41,340
has to be lambda and one,
because that multiplication

803
00:48:41,340 --> 00:48:43,500
will give me the zero.

804
00:48:43,500 --> 00:48:47,240
And this multiplication gives
me -- better give me also zero.

805
00:48:47,240 --> 00:48:48,650
Do you see why it does?

806
00:48:48,650 --> 00:48:52,670
This is the minus lambda
squared plus lambda plus one.

807
00:48:52,670 --> 00:48:56,360
It's the thing that's zero
because these lambdas are

808
00:48:56,360 --> 00:48:56,930
special.

809
00:48:56,930 --> 00:48:58,490
There's the eigenvector.

810
00:48:58,490 --> 00:49:07,900
x1 is lambda one one,
and x2 is lambda two one.

811
00:49:07,900 --> 00:49:12,520
I did that as a little trick
that was available in the two

812
00:49:12,520 --> 00:49:14,130
by two case.

813
00:49:14,130 --> 00:49:17,680
So now I finally have to --

814
00:49:17,680 --> 00:49:20,390
oh, I have to take
the initial u0 now.

815
00:49:20,390 --> 00:49:22,710
So to complete this
example entirely,

816
00:49:22,710 --> 00:49:26,680
I have to say, OK, what was u0?

817
00:49:26,680 --> 00:49:28,740
u0 was F1 F0.

818
00:49:28,740 --> 00:49:40,630
So u0, the starting vector is
F1 F0, and those were one and

819
00:49:40,630 --> 00:49:41,130
zero.

820
00:49:43,910 --> 00:49:47,150
So I have to use that vector.

821
00:49:47,150 --> 00:49:50,390
So I have to look
for, for a multiple

822
00:49:50,390 --> 00:49:56,100
of the first eigenvector and
the second to produce u0,

823
00:49:56,100 --> 00:49:58,070
the one zero

824
00:49:58,070 --> 00:49:58,570
vector.

825
00:49:58,570 --> 00:50:05,430
This is what will find c1
and c2, and then I'm done.

826
00:50:05,430 --> 00:50:10,470
Do you -- so let me instead
of, in the last five seconds,

827
00:50:10,470 --> 00:50:14,920
grinding out a formula,
let me repeat the idea.

828
00:50:14,920 --> 00:50:19,100
Because I'd really -- it's
the idea that's central.

829
00:50:19,100 --> 00:50:21,610
When things are
evolving in time --

830
00:50:21,610 --> 00:50:25,730
let me come back to this board,
because the ideas are here.

831
00:50:25,730 --> 00:50:30,400
When things are evolving in
time by a first-order system,

832
00:50:30,400 --> 00:50:34,480
starting from an
original u0, the key

833
00:50:34,480 --> 00:50:39,956
is find the eigenvalues
and eigenvectors of A.

834
00:50:39,956 --> 00:50:41,580
That will tell --
those eigenvectors --

835
00:50:41,580 --> 00:50:46,710
the eigenvalues will already
tell you what's happening.

836
00:50:46,710 --> 00:50:48,540
Is the solution
blowing up, is it

837
00:50:48,540 --> 00:50:51,390
going to zero, what's it doing.

838
00:50:51,390 --> 00:50:56,240
And then to, to find
out exactly a formula,

839
00:50:56,240 --> 00:50:59,290
you have to take
your u0 and write it

840
00:50:59,290 --> 00:51:03,270
as a combination of
eigenvectors and then

841
00:51:03,270 --> 00:51:05,820
follow each
eigenvector separately.

842
00:51:05,820 --> 00:51:10,930
And that's really what this
formula, the formula for, --

843
00:51:10,930 --> 00:51:15,190
that's what the formula
for A to the K is doing.

844
00:51:15,190 --> 00:51:17,180
So remember that
formula for A to the K

845
00:51:17,180 --> 00:51:21,770
is S lambda to the K S inverse.

846
00:51:21,770 --> 00:51:22,280
OK.

847
00:51:22,280 --> 00:51:24,590
That's, that's
difference equations.

848
00:51:24,590 --> 00:51:33,460
And you just have to -- so the,
the homework will give some

849
00:51:33,460 --> 00:51:41,180
examples, different from
Fibonacci, to follow through.

850
00:51:41,180 --> 00:51:48,900
And next time will be
differential equations.

851
00:51:48,900 --> 00:51:50,450
Thanks.