NARRATOR: The following content is
provided by MIT OpenCourseWare under a Creative Commons
license.
Additional information about our license
and MIT OpenCourseWare in general
is available at ocw.mit.edu.
PROFESSOR: This is my lecture on section 6.1 of the notes.
Really, the central problem of numerical linear algebra is
solving A*x equal b by elimination.
That's what is used much more often than anything else,
but we're talking about sparse matrices, large sparse
matrices.
That's what we have in mind for a long time now --
probably all lectures coming up -- thinking of those matrices.
And this is the prototype of a large sparse matrix.
The main point of the lecture is,
if you take the unknowns in whatever order they're given --
here I took them numbering along the rows -- one, two, three,
four, five, six, seven, eight and so on --
you got to stop and think or get the computer to look
at reorderings, because the reorderings can make
an enormous difference in the fill-in.
So this is, actually, the key word for this lecture, is --
and the key concept is the fact that as elimination goes
forward, a lot of these zeros become non-zeros.
Let me see it happen, first of all.
So this is the whole idea of fill-in.
What's going to be the very first entries that
are filled in?
Let's see.
I guess that the very first row operation,
if you took the matrix like this and did elimination, 4
is the first pivot.
You would add 1/4 of the first row to the second row
to make that a 0, right?
Then what would happen?
This would change that number, but this would put a new number
in there.
So a new number would appear when
we do that first row operation.
And then, also working with this pivot,
I would operate on that row to make this a 0,
and then this would change, that would fill in, right?
I guess that this already has something,
so that won't be new.
So each of those steps, each of those row operations filled
in a non-zero and reduced the sparsity of the matrix,
and the sparsity is what we're living by.
Now if I continue and continue, we'd need MATLAB --
to use spy in MATLAB to see what the picture would look like.
It's very interesting to see it.
Spy shows you the non-zeros.
We're very much focused on what's 0
and what isn't 0 in matrix entries.
We actually keep track of that, the structure of a matrix,
as well, of course, as separately keeping
track of the actual numbers.
One place, we're keeping the actual numbers,
but in the decisions on ordering,
we're only interested in zeros and non-zeros.
So let me look even more at this fill-in.
Now, you saw it happen up there.
Let me just look at a single fill-in step.
So I have a row j and a row i, and this
is going to be the pivot that I'll use to eliminate this,
but in this row, there's another non-zero and in this row,
the k-th entry is not non-zero; it's 0.
So this is what we start with prior to elimination,
just as we did there.
I just want to see -- focus on small --
on the fill-in occurring.
Of course, this isn't on the diagonal.
This is in column k, which is somewhere far
out in that case, somewhere close by in that case.
But we're just -- this is on the diagonal.
That's the pivot, the j-th pivot, that I'm ready to use,
so I subtract a multiple of that to remove that.
So what does -- elimination will not change the pivot row and it
will produce a 0 there, that's the point.
Then the real point that I'm making
is that when I multiple this by the right number
to produce 0 and do this subtraction,
this 0 disappears, that fills in.
That's some a_(i,k) new that -- the new value in row i,
column k is not 0.
so we see that -- as we saw it for numbers,
here's we're seeing it symbolically.
Let me see it graphically, because everybody sort of --
to visualize the big picture.
I think everybody finds the graph
description of the non-zero structure the simplest
to imagine.
So the graph for this little problem would be --
so what's the graph that tells me where the non-zeros are?
Again, it doesn't tell me what numbers they are,
just tells me where they are, which positions.
It's only the positions that I worry about
in deciding on the ordering to use.
Then I execute using the actual numbers,
but in deciding on the ordering, it's just the position.
So I think of my graph as having a node for every row.
So that's node j for row j.
This is node i for row i.
Somewhere we'll have a node k.
So, at the start -- so those are the nodes.
What are the edges?
There is a non -- so the edges come from the non-zeros
in the matrix.
So there's an edge from j to k, because this is not 0.
There's an edge from i to j or j to i.
I'm not worrying about the direction --
this is not a directed graph right now.
So this is the start, but there's no edge here initially,
because a_(i, k) is 0.
Now in this graph world -- it's so small,
we can just focus on the part that's going to change
a little.
The rest of all the other rows have their nodes
and all the other non-zero entries have their edges.
So this is a tiny part of the big graph.
What happens to this tiny part?
So again, I want to see the same elimination step, just
this way, in the graph world.
So after, what edges are there?
So again, it's j, i and k -- and after I do elimination,
j is still connected to k.
That's still there.
No change.
j is no longer connected to y, because elimination
killed that edge, but elimination created
a new edge, i to k.
So that's essentially what elimination does.
Maybe we can say it in words.
When this edge disappears, at the same time
we create new edges connecting any two nodes that
were both connected to j.
You see that point?
If I'm using j as the pivot row and I look on that row and say,
what non-zeros are there?
What edges are connected to j, and I see one -- and then,
when I do elimination, I'm going to introduce,
exactly as has happened here, a new edge between any pair that
were both connected to j.
So we're kind of fighting against creeping filling.
The question is, what order should we
do these eliminations?
So now, think of a larger graph.
Actually, I guess we could somehow think of that --
we can try to visualize it, but maybe not succeed.
So the graph for that problem -- could that serve as the graph?
I guess it could, really.
I would remove -- I'd [? whiz ?] away the --
if I'm given the values on the outer edges,
then I would remove all those, but essentially,
this is the graph for that matrix, right?
If I number each of the nodes 1 to 16,
and I would do the numbering the way I did to produce that
matrix -- now the whole question is,
if I do some different numbering, will --
that'll shift around.
That'll just be a permutation matrix, right?
If I do a different numbering -- so matrix-wise,
I have my matrix K2D -- what happens matrix-wise for --
if I do a numbering, a different numbering, a reordering?
That just permutes the unknowns and the equations.
So really, what happens is some permutation multiplies
K2D on the left and its transpose does the same thing
to the columns on the right, because this permutation
will change the order of the equations.
This permutation will reorder the columns to come back,
to bring the diagonal back onto the diagonal,
but in a different order.
We won't know it with all those 4s, but that will have,
again, all the 4s on the diagonal, but the minus 1s
are moved around.
Of course, none have disappeared or been created by that.
It's just that's giving us a new order.
So that the reordering matrix-wise.
What's the reordering graph-wise?
Graph-wise would be, instead of this numbering of unknowns --
one, two, three, four, five, six, seven, eight and so on --
nine, ten, eleven, twelve, thirteen, fourteen, fifteen,
sixteen -- we have a different order, different numbering.
The computations, the computer science aspect, you could say,
was not going to be difficult.
Actually, we should think and maybe we'll think in a minute,
from the computer science aspect, the actual execution --
I'm probably not -- certainly not going to write out
a permutation matrix.
That's ridiculous.
I'm not going to draw a graph and keep that --
I'll keep some list of positions for non-zeros,
so we'll say a word about that list later.
Here I'm trying to say, how do we
get an idea of what the effective reordering is
and what a good reordering would be?
So here, this is a specific reordering
that I want to mention right away,
because it's so straightforward.
You can get the idea of the red-black ordering
without any big computations.
This is the -- fundamentally, the winner.
This is what people use all the time and MATLAB's backslash
is going to have that available.
Inside this, the codes that are called
are the column or symmetric matrix,
approximate minimum degree.
So that's what this AMD means -- approximate minimum degree.
So that's the basic idea that I'll
come back to as soon as I've just
drawn the red-black picture.
It's so important that people don't stop thinking.
So another sort of high-level picture
of how to do an ordering is look for graph separators.
That's become a big step in the world of computational graph
theory, and that leads to something called
nested dissection, which is separators of separators
of separators, the usual keep going with the same idea
that you see in algorithms.
What about red-black?
Red-black, I'm simply going to imagine that this is
a checkerboard -- it's the nodes that I'm looking at, not
the squares.
So that's red, black, red, black.
So I'm going to number first -- I'm going to take all the reds
first, so I'm changing this number now to take all the reds
-- one, two, three, four, five, six, seven,
eight and now all the blacks -- nine, ten, eleven,
twelve and so on.
I'll stop there so your eye will know how to fill in.
What's the point of that?
If I take a red node and look -- suppose this first one is a red
node, as it is -- then what do I know about the positions
of those minus 1's, the non-zeros in the matrix?
They're associated with black nodes, right?
The neighbors of this node number one are black.
The black ones have numbers from nine to sixteen.
So then the black -- everybody sees it.
With this molecule that has a 4 in the center and minus 1's
beside it, those minus 1's are associated with nodes
of the opposite color.
That's what we see here in the actual numbering.
So what does the matrix become?
What does this -- if I do that permutation,
the permutation that's associated with that
renumbering -- what will the same matrix --
here's the same matrix, but now I'm permuting it and so what
does it look like?
Maybe I can draw it smaller.
I think I probably can.
So in this pretty special situation where the red nodes
are not connected to any other red nodes, only to themselves,
I think I'll only see the 4I -- this is red to red.
Those are the eight red nodes, an eight by eight identity
matrix times 4.
The blacks are only connected to the black and all the minus 1's
are in our connections between red and black,
that is, this far off the diagonal, what shall I call it?
B I'll just pick a letter -- B and B transposed probably,
because I know the matrix is going to stay symmetric.
If it started symmetric, as it did,
then this operation of reordering rows and columns
compatibly is going to leave it symmetric
and it must look something like this.
Is that matrix more sparse than before?
No.
It's got the same minus 1's.
They're just all pushed up into this corner.
Will it have some fill-in as we go forward?
What's the answer on that one?
As I go forward, maybe I should --
I haven't really thought that through.
My instinct is yes.
We'll have fill-in.
Will we have more fill-in or less fill-in than the row
by row ordering?
I should know the answer.
I think I'd probably ask the machine to tell me.
My guess is probably not more and we can --
why do I think probably this is better?
Of course, I should know exactly.
In general, it's better to put off the evil day -- that is,
this goes through the first half of elimination -- I mean,
you know what I'm thinking here.
All the off-diagonals have been sort of
pushed down in the matrix.
Let me do an example that shows why that's good.
Suppose my matrix has a non-zero diagonal -- and by the way,
the first step in the good codes for elimination do
a permutation exactly to produce a non-zero diagonal,
so you're OK to start.
You know where you are.
Now suppose that the non-zeros all show up --
and I'll make the matrix symmetric.
That's the non-zero pattern and my point
is, that's a very good non-zero pattern.
Why is it good?
It's good because there's no fill-in whatever.
When I subtract some multiple, when
I go to make this 0, to eliminate it,
I subtract a multiple using the first pivot.
It changes that, but it doesn't fill in anything new.
The only operations required are operating on the last row,
which is already full.
So there's an example in which the bandwidth -- see,
normally the bandwidth is a good guide --
but that's only for pretty full bands.
Here's a matrix that's not banded, really.
Its bandwidth is the full five.
I have to go out five columns until I get the last,
until I get the non-zeros.
So that is a big bandwidth.
So bandwidth w, the bandwidth w is large, but --
it's the perfect ordering -- but no fill.
I don't fill anything -- you realize that for banded matrix,
we won't have fill in outside the band.
Our concern is for fill-in inside the band and here,
we don't have any, because the bad row -- and actually,
it often happens in some least squares and other problem that
you might -- this isn't quite as stupid a matrix as you might
think.
These might be blocks rather than single entries.
You often want to think block-wise about a matrix.
You might think, why would you have
a matrix that was well separated with block diagonal, which
is really good, and then a few rows that are full.
That seems sort of unlikely, but I guess I want to say it isn't.
If you had a problem in least squares minimizing
some expression involving that matrix,
the block diagonal part, but with the constraint
that maybe the sum of all the unknowns has to be 1,
then when you use a Lagrange multiplier to bring
that constraint into the problem,
into the matrix, that's going to show like that.
And the beauty of the Lagrange multiplier device
is that, also, that it'll produce a symmetric column
there.
We'll see that in the third part of the course, but I'm just --
my point is that this example is not as artificial as it seems.
Now suppose we had taken the bandwidth as our metric,
as our goal.
In reducing the bandwidth as our goal,
we would have made a bad mistake.
If I wanted to do a permutation that reduces the bandwidth,
I better put this up in the middle somewhere.
So this is -- that bad row comes up in the middle and then,
I guess, these are maybe whatever --
I guess I need three X's down here.
I'm going to do the same permutation from the right,
so that will move this column over to the middle.
This would be a reordered -- this would be a P matrix P
transposed thing, I think.
It's got a better bandwidth, right?
Now we have -- we're totally confident of all these zeros
here.
But it will fill in.
Where will we get fill-in, in that matrix?
So as we start, we're using elimination to remove those
entries, no problem, because that upper left corner is
the way this big, the whole matrix was,
no fill-in up to there, but just as soon as we reach this one
as our pivot row and use it to subtract multiples to knock out
those, when I make those 0 by using that pivot,
it's going to bring the whole -- this will be gone at that time,
but this will be non-zero and that will produce new edges
in the graph.
Might be fun to look at the graph that
is associated with this matrix and its permutations.
But anyway, you see that this will all fill-in.
So again, in that small problem, we
could really see what was happening.
In the big problem, we can almost
see it when we take a very specific, highly structured
permutation like red-black, although I haven't pinned down
exactly what the count is like now.
I'd be glad if somebody would just run a computer experiment,
count non-zeros, count -- yeah.
So what's the good thing to count?
How do you see what the fill-in finally
was when you've done an elimination?
I guess that the L and U factors, which
is what elimination produces, that's
where the fill-in will be.
So maybe I just make that point, so that you
could make these experiments that I have in mind
and tell me what I ought to know about it.
So if I have the matrix K, whichever it is --
whether it's this one or this one or any other,
then the effect of elimination is to factor it into a lower-
times an upper-triangular matrix.
I'll just mention a special case while I'm --
special positive definite case --
we can adjust it so that's K is L times L transposed.
This is called Cholesky.
We can arrange the -- it will happen.
We can arrange the scaling so that by putting the square root
of pivots with L and the other square root with L transpose --
so Cholesky is a MATLAB command, C-H-O-L, and actually,
MATLAB -- if you give MATLAB a symmetric matrix --
if you give MATLAB a matrix to work with, it will first ask --
it will ask, is it symmetric?
If it is symmetric, it will hope that it's positive definite,
so it will begin Cholesky.
It'll begin to look for this -- it'll begin the factorization,
which is perfect for positive definite ones,
until it succeeds and finishes Cholesky or until it discovers
that nope, a negative pivot appeared, which signals,
that the matrix is not and was not originally positive
definite and then MATLAB has to accept, OK,
now I may have to do row exchanges,
pivoting operations and all the rest.
So MATLAB's sort of optimistic until it --
on a symmetric matrix until it is told otherwise.
But not all matrices are symmetric,
so this would be the general case, almost the general case.
I haven't included here the pivoting,
the row exchanges that MATLAB does
as it executes elimination.
So let me just finish my sentence on L
and U. L will have the fill in, in it.
L will tell us -- and U will, certainly.
I mean, they both will.
U will have these -- that guy is going to be part of U and this
one is -- I expect to see something there in L.
So L and U will actually exhibit the fill-in,
so that you could find out how much fill in there is simply
by counting the non-zeros in the matrices L and U.
This is to encourage you, because this is really
a pretty experimental subject.
Nobody -- the best methods, which --
or the favorite method.
I think nobody can say best, but this approximate minimum
degree, the favorite method, we could prove some theorem
to show that it was better than something else,
but to really see, is it successful
or not on a wide variety of problems,
that's really experimental.
Just try it.
Now what does it mean?
So I looked at that specific one.
Now I'm ready to look at the approximate minimum degree,
and I'll contrast it with the perfect minimum degree.
There are codes for that and those A's become M's.
But it turns out -- and I don't know exactly why it's the case
-- that the approximate minimum degree usually, if anything,
it's superior.
So these are the ones to use.
So what does minimum degree mean?
Minimum degree means -- let's see --
if I took this problem -- I should draw the graph for this
matrix.
So what does the graph look like for this matrix?
I've got six nodes -- let me put six nodes in --
let me put five and the node in the middle will be the one that
they're all connected to.
In this ordering, those would be one, two, three, four,
five and six -- and all the connections go into six.
So that's the graph for that matrix.
The graph for this one -- so you probably see it better than I
do.
What I had originally there -- I guess this is the graph with
the numbering one, two, three, four, five, six --
what was there before the fill-in happened?
The graph would look the same, but the nodes
would have a different numbering, right?
One, two, three, four, five, six --
so shall I just put the bad guy here?
One, two, three, four, five, six and then what's the point?
As soon as I use number three to eliminate four --
three was connected to five, so a new edge is going to appear
there.
Three was connected to six, three is connected to four.
When I use it, a new four, six is going to appear.
So those are a couple of the fill-ins.
Those are the two fill ins in row four.
Then when I go onto row five -- I'll fill in there.
So those are the three below the diagonal
and above the diagonal, non-zeros that come
from the bad ordering, I think.
I didn't prepare that, so I hope it's right.
Yeah
So what's minimum degree?
Minimum degree is simply sort of a greedy type algorithm.
We have no way to look far, far ahead
and see what non-zeros are coming way, way down the line,
so let's just take it where we are.
We have to choose what should be the next pivot.
Let's say we take that first pivot.
Good.
So what are the degrees of these ?
Let me redraw the graph before the fill-in.
One, two, three, four, five, six.
So I've got six nodes.
That has a degree 1.
It's connected only to one other node.
Actually, all these have degree one
and this one has degree five.
Five edges are going out.
So the whole point of minimum degree is, don't pick that one.
At each step, create the new graph.
The graph expresses a fill-in -- and we're not going to be able
to avoid some fill-in, normally -- but at every point,
take the graph as it is and choose as the next node to work
on -- that means the next row to use in elimination,
the next pivot to choose -- choose one that has smallest
degree.
Don't choose this one.
So for this very special case, we
have five choices for what comes first.
We make a choice like that one.
We do elimination.
In this case, it doesn't fill in anything.
Now I'm looking at the rest of the graph.
I have four nodes there with degree only 1 and a node
there with high degree, so I don't choose that.
So I choose 2 and so on.
So of course, minimum degree, approximate
or perfect or any dumb idea is going
to create the right ordering that produces this matrix.
I hope you see what minimum degree is.
At every stage, you look down the column, just as here.
Actually, what would minimum degree give here?
I would love to have a complete picture.
I think we could at least see at the start.
Let me erase this red-black ordering
and prepare to write in a minimum degree ordering.
If I can re-complete that graph.
So what node do I take first?
Remember, the graph really does not have these pieces.
I might as well erase them now.
Shall I just erase -- these are pieces that connect to known
values that are on the right hand side.
So I think the graph doesn't have those connections,
those connections -- it's going to look pretty much as it did
before, but I think now we have the graph of actual 16,
the graph that we should be looking for.
That's the graph of our matrix, the graph
of the nonzero structure in the matrix.
So what do you take as the first node?
You take the corners, right?
The corners are going to win, because they have degree 2.
These guys have degree 3.
There's a degree 2, so I might take that -- let's see,
what happens?
When I take this one, then I have to see,
when I use that as the pivot -- exactly what I did here --
what's going to fill in?
I guess explicitly I know that two new edges fill in
and what are they?
Maybe they're the same.
I guess they are.
This is edge number -- on the graph,
it's probably easier to see than in the matrix.
So I use this guy.
By elimination, I'm removing the connections,
so it maybe sits there, surviving
to remind me which node was numbered one,
but these connections have been removed by elimination.
That entry became a 0 and that entry became a 0,
but there was a fill in here, right?
That's the first step of minimum degree.
The second step, as you say, I'm going
to do all the corners first, right?
Because that has degree 2 and these still have degree 3.
No improvement.
It's kind of fun to do.
Two and that filled in.
Number three and that filled in.
Number four and that filled in.
What next?
There are going to be a bunch of choices now.
As far as I know -- I could be wrong --
I don't know whether the -- I don't know what the choice
decision -- how the decision is made when there is a choice,
when I have one, two, three, four, five, six, seven,
eight nodes, all with degree 3.
Let me just take this one.
So I'll take this as the next one, as number five, I guess.
That'll be number five.
That will kill all those connections
and just leave the node, but it will put in that one.
Any others?
Maybe only that one.
Did I do that right?
I mean, it would have put in this one,
because those are both connected to this,
but that edge is already in there, so nothing new.
Actually, here I've created a node that is also degree 3.
So I'm going to have a lot of degree 3's here.
But then, after a while, I think when I get into the center --
I'm guessing that the degree will go up.
I don't think I get away with degree 3 all the way
until the finish, I don't think.
Anyway, you see the point that there
are choices to make, among nodes with equal degree.
Then there's this calculation to make -- maybe I should --
this is a moment to just mention, if I can do it,
how you might keep -- how you might organize this pass
through -- so this pass through the matrix --
it's really a just a pass through the graph.
It's not looking at the actual numbers.
It's just choosing the ordering and then the next pass
through we follow that ordering with the numbers,
finding the multipliers that go into L
and the numbers that are left in matrix U.
So how would I -- how would anybody reasonably write down
the original list?
So I have the nodes numbered one, two, three, four --
up to sixteen.
Those are the node numbers.
Now with node one, I'll list all the --
so this corresponds to row one of the matrix,
so I'll list all the nodes that are connected to row one,
to node one.
Node one itself is -- I don't know whether to put that
in or not.
Maybe I will.
Number two is connected to it and number five above it.
I'm taking the original row ordering.
Then when I look at node two -- this is going to be a long
list, but I'll have to have a break there to tell me that
these numbers were associated with row one.
Now I want to know those associated with row two.
So row two had the connection back to one,
so this little group is going to be all the things connected
to row two.
Remember -- sorry, since I've erased all those connections,
but you remember them -- two was connected to one,
it's connected forward to three and above it to probably six.
Maybe that's one, three, and six.
And then a slash, and then whatever node three --
so I need -- I guess I need one more line there to tell me --
I need a pointer.
I need a line, a pointer that tells me, number one,
start with the first one -- one, two, three.
Number two, start with the fourth one.
Number three, start with the seventh one.
So that number seven is pointing to the beginning
of the sub-list, the beginning of the short list
that's associated to each separate node.
I guess what I'm saying is that this
is a pretty compact description of the graph.
This will tell me all the edges in the graph,
It tells me what two is connected to
and this tells me where that little set lies.
Now if I, as I will, change the ordering,
then I just reorganize the pointers.
I'm going to speak roughly and not
the details, just in general, in the large,
you could imagine that this is a pretty convenient structure
to use for keeping the record of what the node numbering is.
The command nnz is a useful one.
That's a MATLAB command that counts the number of non-zeros.
So n, n and z suggest exactly what that will do.
So if you pick an ordering, like approximate minimum degree,
and you pick a matrix, like this one, and you run --
I've been very interested in that.
You take the matrix, K2D, you take the permutation
that comes from approximate minimum degree, that gives you
this reordered matrix.
We then do elimination, which factors it in L times U --
probably L times L transposed, because it's a positive
definite.
Then I would do nnz of L as a measure of fill-in.
That would count the fill-in in L,
and with symmetry, it'll be the same in u.
If you have a better idea than approximate minimum degree,
that's a measure to use as the criterion.
I've just a little time to speak, finally,
about this idea of graph separators.
So again, I have to get back to a graph.
Let me draw a graph under.
So now I'm ready for this one.
I'll put a little star next to it,
to say that it didn't look for a long time is
if it could compete with number two,
because it'll be a different ordering.
You could, in a way -- it's going to be a block minimum
degree order.
Let me say what it is and as I draw the graph,
let me say when it might win.
It might win when the size goes way up, 3D.
In 2D, minimum degree is -- on sparse matrices,
is quite successful.
So in two dimensions, until the size really gets big,
that's going to be fine.
In three dimensions -- I'm still going to draw only two,
but in three dimensions or very, very large matrices,
I want to think about this graph separator.
So let me -- I'll make it just so I really separate it
perfectly -- let me make the number --
so when you look at that graph, you might say, if I --
look at that set of nodes.
So a separator is a set of nodes that
separates the rest of the nodes into two groups.
So I have a group of nodes P, a group of nodes Q
and nothing, no connections between P
and Q. P is this part.
Q is a similar part over there.
S, the separator, is the part with the connection.
So P is connected to Q only through S.
So now if I think of ordering, in what order should I
take the P, the Q and the S?
Our absolute rule is, everything's connected to S,
put it last.
So you see that this is the block form of my small example
here.
I didn't particularly have a P or Q,
but I certainly wanted to put S last.
Maybe I did.
So maybe the separator here was -- so what's the separator?
One possible separator would be like that, as S,
P for these two nodes, Q for these three nodes,
and the rule would be -- I haven't said what I'll do
within P or within Q or within S,
but the rule would be put Q last.
Of course, that was exactly the right thing to do.
We're following the same rule, just block-wise.
So we have the -- so our matrix --
what does our matrix look like when you choose this ordering?
You have whatever P is, the block of all these P's.
You have the block of all these Q's, which
might be connected to each other and are, but not to P.
So the main point is, you have these 0 blocks.
And then everything is connected to S
and S is connected to itself.
So all you've achieved, but it could be considerable,
is to take an ordering in which P was --
since P is not connected to Q, that produced these large 0
blocks and of course, they'll stay 0 as elimination goes
forward.
those 0's there will -- I don't have to do any pivoting
operations, any row operations to make those 0.
They're already 0 and so the only operations
will be P onto S, Q onto S, S onto itself.
In many problems, it's the S onto itself part --
so what will happen then, when I do elimination on S?
When I do elimination -- I'll have whatever work is necessary
within P, so I'll be factoring -- can I call that P,
that part of the matrix?
It's an abuse of notation.
This is the connections of S to P.
This is the connections of S to Q, S to itself, Q to S,
P to S -- probably symmetric.
So when I do elimination -- if I don't, if I accept this
ordering -- if I accept this ordering and just go for it,
elimination will factor P into L*U --
because it'll work up in the corner as it always does --
then there's no connection here, so it'll factor Q into its own
LU, and of course, all that -- all this stuff down here will
change, but effectively these will become 0's, because then
there's a serious elimination, P onto S, will make that 0.
Q onto S -- Q will make that 0.
This will be some -- I'll call it SSS.
It's a much fuller matrix, block -- smaller though,
hopefully smaller -- but pretty full.
So actually, in the graph separator nested dissection
world, most of the time is spent on that third block,
just working with it, because it's
the one that's pretty full.
So how do graph separators work?
First of all, you need an algorithm
that finds the separator and a lot of computer science
attention has gone onto that.
Given a graph, cut it in half.
We don't want more nodes in the separator
than necessary, because the work is going to end up there.
So we want to cut the graph pretty nearly in half
by a short separator.
If we only did the dissection once, that's what we need,
but this idea of nested dissection --
you know what's going to come before I say it.
The idea of nested dissection will be, take this,
take the graph -- can I just draw it this way without
drawing all the edges?
Cut it by a separator S_1.
That gives a P_1 and a Q_1.
But now, what's coming?
You're going to separate P_1.
In order to do the work up in that left corner for the P
matrix, that's itself a sparse matrix, so now
I'll look for a separator and what separator
will I choose for the left side of this graph?
I'll choose the horizontal separator.
This will be the second separator.
It'll separate the two bits of P.
The third separator will separate the two bits of Q.
Now I have this part to work on, a fourth separator will --
you'll see its a nested.
These separators become nested.
There's S_5, there's S_6 and so on.
I think everybody sees in principle
what nested dissection should do and the question is, how does
it compare with minimum degree?
You see, it's just like a different --
we're off on a different route here.
You can compute asymptotically for large N,
what power of N appears there, what power of N appears
in minimum degree -- but I would love to know, does the --
I would be happy if somebody used approximate minimum degree
on our model problem and maybe also on some other problem.
We could see what power of N -- that's essentially what
we're looking for.
So it's N to what power for fill-in and for the operation
count, which -- the number of flops,
which controls the cost and the time.
So it's what power of N appears there
and what power appears in any other method.
There you have it for direct elimination methods.
I'll just mention the name of the --
guy's at University of Florida, and he's the one who
contributed the code that MathWorks uses and has just
announced a book coming on this exact topic,
to come later this year from Sci Am.
So there will be a book from the person
I take to be the leader in this algorithmic development.
Just to remind you, Friday is my guest,
former PhD student who is going to speak about numerical
methods for finance -- Black-Scholes equations
and other problems to find the value of financial derivatives.
See you Friday.